For a start, please read http://www.mnot.net/cache_docs/
for good hints on how the different http headers plays together. There are plenty other articles on the net about html and caching, so it is worth spending a few minutes to study them.
When you have done that, then you can "fix" your site by adding a response rule (not requst rule!) that adds the required headers (Expires and Last-Modified) to the reponse if the url matches a pattern of if the content type is of certain types.
Did this help you along or do you need more specific guidance?