Yes, investigating the other lifepods is a good idea. Might gain a blueprint or two along the way. It's a mechanic to introduce you to various biomes, so new places to see, new creatures and most importantly, new resources!
Picking the first base location... so many possibilities. In both of my playthroughs, I built the first base close to the pod. First one directly below it, and the second about 50 yards away. As long as you stay in the shallows anywhere is fine, as you can easily get solar power. Any deeper though and you'll need a different power type, and since those blueprints probably haven't been located yet, the shallows makes sense.
I tend to look for some sort of shelf with deeper water next to it. Eventually you'll want to build a moonpool, and having it over a bit of water is good. If you plan to build a single base, consider building near a thermal vent. You can use solar now, and add thermal later to support all of the whizbangs your base will have.
A good view is paramount for me. The vista is a primary consideration. To the northwest there are nice cliffs beside the mushroom forest, with a thermal vent nearby. Never built there myself. Or to the SE where the Grassy Plateau, Kelp Forest and Shallows meet is another (with thermal vent) This was the spot where I built my second and main base in the first run. A nice location with deep water access for late game. I also built here in the second run, but then disassembled it to rebuild in the pink cave shown in the second shot. I try to choose a spot not near any stalkers or gasopods.
If you do build with a view to thermal, be aware that there are power relays that can be built, so the base can be quite some distance from the actual vent.
This was my first base (run #1) before the renovation and expansion
And this was the main base from the first run. More was added after this shot, including a second moonpool. This is close to the SE Shallows thermal vent.
It has all of the stuff you can build, plus two separate stacks of multipurpose rooms on the top level, each with one room completely above the water so I could dry out from time to time
Another shot, along with the toys, from the other side in daylight. The second moonpool had been added by then