During the POW rescue, the Cap has a knife in a sheath strapped to his left ankle:
This may be an M3 Paratrooper knife and scabbard, but with an additional strap to tie it to his leg. Since the ‘pocket’ tie on the Cap’s left leg is missing, this might be what it was used for. Conversely, after the rescue the knife switches sides:
The above pic shows something that looks more like leather, tied lower on the scabbard as well.