I was patrolling the North Sea close to the coast of Norway and had visual contact on a coastal merchant. I got into position and fired 3 torps, all hits and she sunk. Turns out, she was from the fatherland. I got closer while she was going down and was able to make out the flag, however, that was the only way to tell. A coastal merchant from the allied side looked exactly the same.

Is there a better way to ID without getting close enough to see the flag? i.e. silhouette?

I think all coastal merchants(and all other merhcants as well), regardless of nationality, has the same shape. So you won't be able to ID it's nationality by silhouette. I don't know what realism level you're playing on, but i(a noob)play at very easy as of yet, and the method i use to ID nationality is by the map. Red for enemy, Green for neutral, and friendly i don't know yet since i've never ran into one.

Friendly is blue. The only way to tell outside of using the map updates is to check the flag.

Welp, good to know. Looks like I'll have to get closer when it's a coastal merchant. Or save it right before I sink her.