intimidating my co-workers, one crazy patron at a time

so we have this patron, E.J., and he calls the library no less than 30 times a day. usually he is asking the same question over and over and over again. so he called me today to ask if his book is in...it went something like this:
EJ: i got a call that my book was in? do you have it there?
Me: yes, that's why we called you
EJ: i ordered a GED book with a CD-ROM?
Me: yeah, it's here
EJ: is the CD-ROM there?
Me: yeah. the book, which includes a CD-ROM, is here
EJ: so it's there and i can come pick it up?
Me: yes. when we call you and tell you a book is here for you that means its here and you can come pick it up

and on and on and on and on and on and on

so when i finally get off the phone with the lovely EJ i make a comment in passing to some of my fellow staff members saying that although i am not a violent person, if i could get away with it, i would have no problem bludgeoning EJ to death with a large reference book. is that an innapropriate comment to make in the workplace?