Oh yes, this is a wonderful area to explore, and Primal brought me back to loving food again rather than the love/hate relationship I'd had for years.
Branching out diet-wise. I rarely eat fish. Liver and offal have been no-go areas so far. Meat and veges come from the supermarket for the most part - not my neighbour's farm or my own garden. I'd love to find a source of organic whole milk.
Fish for beginners. Salmon is very mild and one of the best sources of O3s out there. It can be baked or poached; some people grill or pan fry the fillets. Though it can be dressed up, just a little lemon and butter is really all it needs to be scrumptious. Dill and sour cream or yogurt also work. Salmon also comes in cans, but once you have fresh, it's hard to go back to canned. (Like the difference between sushi grade tuna and that nasty tuna in cans.)
Oysters. The only time I'll eat these raw is when I can watch them being shucked and put on my plate, but that's spendy even if it is a good time. Very mild and perfect with a squirt of lemon and some hot sauce. When I buy them at the grocery store already shucked, I either lightly dredge them in a primal flour and egg and then pan fry them, or I make a chowder of some kind. Very decadent stuff. If you can find smoked oysters in cans, they're wonderful, but I can't find any locally that don't have weird ingredients.
Shrimp and crawfish - I think in NZ you all have the larger crayfish. Find a recipe for "fish boil" or "shrimp boil" which is just a big old pot of water with a lot of spices. Once it's boiling, put in your fish. Remove shrimp when they turn pink, craw/crayfish when they turn bright orange/red. Serve shrimp with melted butter or homemade mayonnaise; if the crawfish are spicy enough (I like them to make my eyes tear), just start pulling off the heads and eating their little bodies. Do be brave and suck out the heads - sounds gross, but it's like some natural spicy sea butter. If you want to use shrimp in stir fries, you have a bit of cleaning to do (or pay extra and buy them cleaned), and put them in after all your veggies are moments from done - once they're pink, they're done.
Sardines. If you have access to fresh, I envy you. Here, fresh are stupidly expensive. Most fish in cans (except for our American pasteurized tuna) are pretty intense, so to start you might want to use them as toppings or side dishes with blander foods. From what I've read, the little briesling sardines are more intensely flavored than the bigger pudgy ones. Sardines are also a good source of O3s.
Herring. I don't see this mentioned on the forums too much. You can find these already kippered or pickled. And while I don't know if this is a universal thing, pickled herring in sour cream is food of the gods.
I haven't explored offal too much yet because I've been comfortable just eating chicken livers all my life. Chicken heart and gizzards are a big thing in the South, where they bread and fry them. I have to say that after a few drinks, deep fried gizzards are very nice. I do have a cow's tongue (not really an organ, but still considered offal) and a beef heart in my freezer. As soon as my pickling spices arrive, I'll start on the tongue. I'm squeamish about the beef heart, but I'll get over it as soon as I find a recipe that appeals.
Good luck on your new explorations and adventures!
"Right is right, even if no one is doing it; wrong is wrong, even if everyone is doing it." - St. Augustine