Setting Update object position to FixedUpdate object position (js)

So I have a strange problem.
My game is in first person, and uses a rigidbody (cannot use a char. cont.) for the player.
The only problem is, that I want the player to use FixedUpdate, but I want the camera to follow as if it wasn't using FixedUpdate, as making the camera follow an object directly that is using physics makes it look like the framerate is lower (since FixedUpdate doesn't update as often as Update).

Here's an image to illustrate what I mean:

Where A is the player (with the dots being the position as it moves over time in relation to Update)
And B is the camera (with the line being the desired position as it moves over time in relation to Update)

But obviously, setting the camera to follow the player directly (like making the camera a child of the player) would make the camera's movement the same as A, which is undesired.

Has anyone else run into this problem before?
Is there even a solution to this problem?

Any help would be great, thanks!

 

Turn on interpolation for the player rigidbody.

--Eric

 

I tried that, but things still look jittery (especially noticeable if for example, looking at an object while you walk in a circle around it)

The controller itself is not jittery if you move the camera so that it is behind it, but the environment is, and only when moving. If you stand still and look around, it is fine.

EDIT: With no interpolation, it's a constant jitter.
With interpolation, only some of the frames appear to be "off".
Extrapolation yields similar results to that of interpolation.

 

Any other ideas?