Thanks for posting video - it makes it so much easier to see your workflow and the problem being encountered.
There are a few issues at play here....
1) The material pick problem: The message being displayed is saying that the geometry you clicked on does not have an
Octane material - so it might have a
Rhino material assigned.
2) The second video diamond material rendering artifacts - this is potentially a ray epsilon issue. (see
http://render.otoy.com/manuals/Rhino/?page_id=476). Those diamonds are very small - so I would think you would need a ray epsilon of about 0.00001.
3) Instead of creating Octane materials one by one and assigning them to layers - rightclick the Settings tab and use the Convert All Materials To Octane Materials.
4) When you drag and drop a material to geometry that has material assigned by Layer, then the material tag assigned to that geometry changes, so you will need to do a Refresh for Octane to get the new material tags.
5) For the last issue in the second video (assigning the Octane material to the layer only applying to one of the gems)...If would be easier to work out what is going wrong if you change the purple Octane material such that Rhino preview window render showed it differently to the diamond material (then you could see if the Rhino preview window was correctly render the newly assigned materials). At the moment you do not know for sure Rhino is assigning the material to the layer correctly.
Sometimes it's quicker to solve these sorts of issues by sharing your desktop with me via Skype. PM me if you want to do that.
Thanks
Paul