If you want to avoid the signature staying in memory the next time that you're trying to apply an electronic signature, untick "Save Signature", found in the bottom left corner of the signature dialogue box. This will eliminate the issue of your signature kept in memory and associated with that document. You will be able to delete the signature text frame that contains the signature. That is unless you save the document and then you'll be able to right-click on any area of that page, and select "Edit Text" from the context menu.
You won't be able to remove the signature because once it is applied it locks the document and it won't allows you to do anything after you click outside the signature box, and click Close to finish the Fill & Sign portion of this wizard. ++Adding to the discussion and the valuable guidance of Akki,