actually, never trolled the harbor, just drift. movement is a little erratic when drifting. usually use live bait, but i had some luck drifting with a 5" sexy smelt hammer with hot sauce carolina rigged with a 3' leader along the jetties. on a slow day in newport, even at the bait barge, i was able to hook 3 keeper calicos and a small halibut in about 1.5hrs on that setup.
btw, if you'd rather use live bait, the m street pier i mentioned is a hot spot. always bait fish there. shiner perch are always available, along with smelt, chovies, sardines, and even the occasional mackerel. i've seen guys throw cast nets, but i believe that's illegal south of point conception. bring a few slices of bread, chum the waters, and drop a sabiki down just below the surface (size 14 or sabiki size 3 is good). if you want perch, drop all the way down next to the pilings after tipping your hooks with shrimp. spending a few bucks on shrimp and bread could save you plenty of dough at the bait barge if you have the time to catch bait.
i've had days where bass and halibut will inhale the smelt/chovies. action on perch is a little slower, but tends to get bigger fish. they also stay alive longer in your bait tank, and you can cast them many times.