I tested this on armor only, but I see no reason why it wouldn't work for any item if it's large enough for your atavism to collide with it.

First off, go to any armor shop that has a piece of armor on display that you want. Jump up onto the shop keep's counter and just run into and knock the armor item off the counter. This is not illegal and noone will even care. Next, keep kicking it by running into it multiple times until you have the item in a corner away from the view of the shop keeper.

The shop owner will actually follow you around the shop while you run about. Get him/her to follow you to the opposite end of the store, then quickly run back to where you placed the armor item. Use the sneak ability and pick it up before they see you. Done.

Submitted by Laerun