No, the poll worker would put the voter's smart card into their nifty little spyrus, and see that it wasn't voted. No problem. (Ken)
What happens if the voter simply returns the voter card, and goes home for lunch before coming back. Or could leave the building to re-read a measure and come back 3 hours later. Where's the voter card at that point? Long gone.
I guess that means you'll also put the "status" of the voter card into VCProgrammer using the Axion and the Journada as well. I think we are in some agreement, but implementing a "non-hidden" button seems to open the door for the voter to cancel without informing the pollworker - unless there was a PIN implementation (see last email on my views there).