First, I 2nd randy's call on the stick and rudder book.. very nice,
very much a classic and gets you thinking of how things work.
I also 2nd the E6B wheel.. you can by one rather inexpensively from
SPORTY's or a local aviation supply store, the manual that will come
with it and is pretty easy to read and figure out with a little attention
paid to it. Kindof a neat tool really - hard to imagine its so old at
this point. Now this is fine, but after awhile you need to get onto the
weather and other fun stuff other than math..

so...
3rd.. one thing I recommend, is get Jeppesen's private pilot handbook,
its pricey but sometimes can also be found in used bookstores. Its a
thick beast but chocked full of info thats meant to be readable. New
its almost $80, but I've seen used one's for 50. recently.
and if you want to learn the Regulations, you'll eventually get a copy
of the FAR/AIM (federal air regulations/airmans instruction manual)
that is the gov. book of rules followed by the airmans handbook..
now its not really the most readable beast but it is full of info.. and its
not that bad 10-15$ or so.. guaraunteed to put one to sleep.
http://www.sportys.com or
http://www.avshop.com are a couple
of folks I've done business with in the past and they've been fine.
ps. the FAA at one time had some useful books online in adobe acrobat
format (.pdf) that you could download for free (and th at you can also
buy a paper copy from the folks above as well). These handbooks
were actually pretty well written and there's one for private, instrument,
and other kinds of study. I had these on my laptop at one point and
read them on some of my trips. I used the instrument one for helping
me with my inst. rating and it was pretty well done.
hope this helps,
henning