Yep, and a base can be as simple as a tube or two with a solar panel for power. Of course once you see how it works you'll want to design more ambitious habitats. But that's enough to get started. If you see the first screenshot I posted, that's a decent sized one, but more than you need really. The mutlipurpose rooms (the round ones stacked together) are the best, but the blueprint is harder to find.

That base has 5 solar panels (each can store 75 power). You just want enough panels to keep the base from powering down while the sun is set. But even if it does power down for a bit it's really no trouble. It will come back online when the sun comes up. It has a moonpool, which can dock a Seamoth or a Prawn, and recharge it. A fabricator, a radio and maybe a battery charger when that was taken, about 10 hours in.

Alternatively, you can build near the thermal vents. There is one in the SW corner of the Safe Shallows and one on the north side of that biome, near where it joins with the Kelp Forest and the Grassy Plateau. Thermal plant blueprint can take some searching. But it's my favorite type of power since it's plug and play. Or is it fire and forget

In the Safe Shallows though, solar is fine. Once you go deeper you'll want an alternative.