Cover potatoes with water in a medium pot and bring to a boil. Simmer until tender, about 10-15 minutes. Scoop out and put in an ice bath to cool them down.
Put shrimp in the hot potato water, bring to a simmer and cook until just pink, about 3 minutes.
Cut the potatoes in quarters or halves, and the shrimp into bite sized pieces. Put in a bowl with all remaining ingredients and toss. Add salt and fresh ground pepper to taste.
Can be made 6-8 hours ahead and chilled before serving.