Smooth Rotation issue

I'm attempting to move the camera around an object slightly, to give the look of changing direction without changing the object's rotation. Code I have thus far is:

Code (CSharp):

1. void FixedUpdate(){
2.         float turn = 0, dip = 0, yaw = 0;
3.         turn = Input.GetAxis ("Horizontal");
4.         dip = Input.GetAxis ("Vertical");
5.         yaw = Input.GetAxis ("Yaw");
6.         transform.localRotation = Quaternion.Lerp(transform.localRotation, Quaternion.Euler (dip * -DipSpeed, yaw * -YawSpeed, turn * BarrelRollSpeed), speed * Time.deltaTime);
7.         //transform.localRotation = Quaternion.Euler (dip * -DipSpeed, yaw * -YawSpeed, turn * BarrelRollSpeed);
8.     }

The code works fine if I only use a single input on an axis at a time. The camera rotates a little, and when released it moves back into position, as expected. The issue I have is using multiple inputs on an axis, such as switching directly from left to right. In this instance, instead of moving smoothly from the left rotation to the right rotation, it starts rotation from the neutral position, say origin to right rotation instead. This is causing camera jumps and is otherwise unnatural.

The commented line, when replaced with the other transform.localRotation line, performs in the exact same manner as far as I can tell. What could I do to get a smooth transition from left to right, instead of center to right?