Nav Mesh Builder

Hey guys, wondering about modifying a big terrain navmesh.
The builder works perfectly when creating multistory buildings procedurally, but the buildings are created in the world which consists of a big terrain, if the builder rebakes the whole terrain + new building each time a change is made it lags and gives a huge profiler spike. If the change is local in the local nav mesh builder - no hiccups, but the mesh builder creates a new nav mesh on top of the terrain navmesh so agents are having issues travelling to it, as there are 2 nav meshes on top of each other. Is there a workaround? I have tried adding 16 objects that split up the whole terrain into local nav mesh builders, but this makes the game load far longer than usual and + AI can not traverse between the local meshes unless I add jump links which is not ideal.

Hitting a wall here :/