Not easy indeed to handle instances with Blender.
As Yoyoz pointed it I think we should avoid using modifiers as they just can't be transported to Octane.
I made several tests and the only thing I could do for now is exporting particle instances matrix coordinates in an external file to be pasted in the scatter node, work pretty well but it really requires some work to fit a complete scene with trees, bushes and whatever requires instances.
By the way here is a quick test :
The workflow :
I export my scene and during the export process the instances matrix coordinates are stored in separate files with the instanciated object as name.
I load the scene into Octane
Add the meshes to be instanciated in separate nodes (they should be exported individually)
Create the scatter node linked to each object
I copy the content of the file containing the matrix and past it in the scatter node for each instanciated object
Create a Geometry group node to mix the main scene and the instances
That's it ...