You use kick yourself in expressions such as I could have kicked myself and you're going to kick yourself to indicate that you were annoyed or are going to be annoyed that you got something wrong.
[feelings] I was still kicking myself for not paying attention.
I immediately regretted having said this–I could have kicked myself.