I think if saying "I thought we decided not to do this?" isn't effective, I'd second what WZ said, to bring this up to management as something for an office meeting. And also, Marianne's ideas for making a friend who is truly special feel cared for are excellent ideas.
I worked at a place for two years where no one seemed to know when my birthday was, but we celebrated everyone elses. It kind of turned me off of birthday-at-work deals. On the third year when one of them discovered when my birthday was, I think she realized they had forgotten me twice and she bought me a cake all by herself despite not really knowing me all that well, real fancy one! I think that behind my back, she chewed them all out because everyone made a huge point of asking me about my birthday (it had been over the weekend that year). It was just an oversight but made me feel bad when I saw what they did for other people (this was my last, crappy job). I faulted the person in charge for not making everyone's birthday known. There is no reason why work places need to be turned into social hour for every holiday and occasion under the sun. I didn't care about a stupid cake, but did care about being left out. At that particular workplace there was no official policy about it and there should have been.
My favorite policy about this was the place I worked at which the employer's assistant bought a funny card and a simple cake, anyone who wished brought a pot luck snack for everyone, and no one was allowed to bring birthday gifts to work, just food. If someone was "popular", we all had more to eat. This took off the pressure and we all enjoyed each other's day and bonded and that was pretty good PR for a five dollar cake and a two dollar card investment. Course it is a little more now, but still. There were about 25 employees there, so there were just too many birthdays to try to share them all or buy gifts.