You must understand that this file only has RGBA encoded color values. Any change on it is going to break this encoding. Let say that you have an instance with a (1.0, 0.5, 0.2, 1.0) RGBA value. This value is encoded into an integer value (that is the variable that Octane understand) that is referencing a single pixel in the .ppm image, that has this color/alpha. This is what Octane can read, the value of this pixel to colorize the instance.
We can't store an unlimited number of colors in the .ppm file ... it is not a matter of the file format or the bit depth. It can be an EXR file if you want, but a 4k image only can store 16M combinations of discrete RGBA colors meaning that you have a 6:6:6:6 bits encoding. We could support 8:8:8:8 bits encoding, but then the texture file should be a 65k image, something that is going to eat much of the available GPU RAM.
Computer Graphics Software Developer
Octane plugin development and support: LightWave and Houdini