Preheat the oven to 240°C/200°C fan.
Pat the lamb shoulder dry using a paper towel. In a small blender add half the rosemary, garlic, anchovies, capers, lemon zest and juice and half the olive oil, Blend to make a rough puree. Pour into a squeezy bottle.
Using the tip of a small sharp knife, make about 20 incisions into the lamb. Insert the tip of the squeezy bottle into each incision and squeeze a little of the mixture into the lamb. Rub the remaining oil into the lamb and season with salt and pepper. Insert a few leaves of the remaining rosemary into the slit, pushing it in using the tip of the knife.
Place the lamb on a roasting tray and cook for 20 minutes until the skin turns golden. Carefully cover with aluminium foil. Reduce heat to 140°C/120°C fan and cook for a further 4-5 hours or until the lamb is soft. Turn off the oven. Pour off any liquid in the roasting pan to make gravy. Return lamb to the oven, cover, and allow to rest in the oven for 30-60 minutes before carving. Serve with beans and steamed new potatoes.
