Souvenirs at the temples are often for sale in Siem Reap to a lower price. Most items can be bargained with. Do not push it to much as wrongly accused by tourists the price are overpriced. It happens but this should not be your standard approach when bargain with the shop owner.
A simple tactic and ensure to get a good deal, bargain with the shop staff a bit, walk away and check out the same item at a different shop and then go back to the first shop. Then you know you get a good deal and a fair price for you and the shop..
Phsar Leu Market at Road 6
This is the main market for locales in Siem Reap The market is enormous, and has everything you need for everyday living. Prices are lower here than the other two markets in town. Knowing some Khmer numbers comes handy because when asking for a price, most likely the price doubles.
You might not find all souvenirs you are looking for and find at the local market around pub street, but it is a great experience to see how the locals are shopping.