That certainly wasn't always true. Back in my high school and college days, I did manage to have friends of approximately the same age. (Actually, they tended to be a year or two older, since I had skipped a grade in grade school, but close enough.) But something happened since then.
Marian said that I mostly like to flirt with women who are "young, pretty, and available". Although there is likely a certain amount of truth to that, it's not the whole story. After all, the older women in my social circles, especially the SCA, aren't as available for flirting with; they are in committed relationships with other people and have mostly given up that sort of thing. So now we start to get to one of the real reasons; available is the key variable here. Not that I object to young or pretty; they have their charms. But I am also drawn to many people who lack one or both of those attributes.
I believe that the real reason for this age gap is that I lack cultural idenfication with my age peers. Again, I certainly had it once upon a time; in the 60s and 70s, we listened to the same music, had a lot of the same values, and so forth. But things changed along the way.
Part of it is that I haven't grown up in some ways. My age peers have gotten involved with houses, marriages, and families. I haven't even felt any significant desire to own a house (it involves more responsibility and rooting to one place than I have yet felt. On the other hand, I do like The Buttery a lot, and I may be responsible for keeping it together someday...), and I don't have any children (nor am I likely to in the future). I am married now, but I don't see our relationship as a traditional marriage, and it just doesn't feel (to me) like I imagine a lot of the marriages that I see feel to the people in them.
Another part of it is that I haven't stayed frozen in time. For example, "classic rock" was presumably invented for Baby Boomers like me (I definitely count as one chronologically, even if I don't identify). But I can't imagine anything more dull than listening to that music as a steady diet -- been there, done that. One must continue to add new sounds to one's musical repertoire; the alternative is a slow death.
In some other ways, I guess you could say that I have stayed frozen in time. The 60s were a time when a lot of values shifted. People spoke of them as a time of "free love", for example. I believe that there is at least some truth there that should have been kept; sex is not an earth-shattering thing that should necessarily be kept for a one-and-only-true-love, but an experience that all good friends should be able to share. People should also be free to have casual sex when they want to; it may not be fulfilling in the same way as sex in a more committed relationship, but it does have unique thrills of its own.
Similarly, I can't support the "just say no" movement. Yes, I did try some drugs (marijuana and LSD) in my youth. Although I haven't taken any illegal mind-altering substances in years, I remain open to the possibility of trying them again in the right time and place, with the right people, and the right reliable source (a big problem, since I'm no longer in a well-connected community of people like I was at MIT). I learned some things that remain with me from my acid trips, especially an appreciation of color and a then-new-found freedom to enjoy touch. To say that taking drugs is categorically wrong would require that I deny a part of myself. I can't do that.
(Some things to mention here. I will never smoke marijuana again; I never cared for the experience of smoking. But brownies are another story. If I trip again, it will be with a close partner; one of the key insights from my last trip is that that is the next place for me to explore that way. I don't know if it would actually be possible to fuck on acid, but the foreplay would be glorious!)
Finally, there is my self-concocted identification as polyaffectionate. I need to have friends in my life that I can share hugs and cuddles with, as well as talk. All too many people of my own age seem to be closed to such experiences with people other than their spouses, or have simply given up such things altogether.
I'm a contact junkie; I need a lot of touch from a lot of people, and
have difficulty feeling close and connected to people unless I can experience
contact with them. (Gee, maybe that's why words like "close" and "connected"
have both of those meanings.) So I tend to spend a lot of my time with
people who can fill that need. The SCA (though, at least around here in
Carolingia, it seems to be the younger generation), contra dancing, and
science fiction conventions are all things that I do (in part) because
they offer a lot of opportunity for friendly touch.