The answer is yes, but it really depends on your setup. You seem to be a bit unclear on networking. For the most part (except for exploits), you have to open up your computer in order for others to access your data.
If TrueCrypt is mounted, anyone accessing the computer can see the data in the clear.
In terms of networking,
1) If you let people log in remotely, they can see the data.
2) If you don't let people log in remotely, but you share the mounted TrueCrypt volume, they can see the data.
If you don't do these, they can't access the data.