Early 20th century; earliest use found in Army & Navy Gazette. Probably arbitrary (or perhaps from the initial letter of query), probably after slightly earlier use of P-boat for a patrol boat.
decoy from mid 16th century:
A decoy was originally a pond with net-covered channels into which ducks and other wildfowl were enticed to be captured. The wildfowl were attracted by a decoy duck, a tame duck trained for the purpose or an imitation duck placed on the water. Decoy dates from the early 17th century and probably comes from Dutch de kooi ‘the decoy’, the second element of which goes back to Latin cavea cave.