Points to Scatter.lua- The script
- (130.6 KiB) Downloaded 80 times
- Rendered with Info Channels - Diffuse
- KingsCollege.txt
- point locations and rgb values
- (30.66 MiB) Downloaded 82 times
- Data Attribution.txt
- Acknowledgement of dataset authors
- (932 Bytes) Downloaded 87 times
Another nice data set can be downloaded here
https://dronemapper.com/sample_data/
For those of you who work with Scatter nodes and point clouds in particular, this .lua script can help.
A detailed description follows with a sample dataset to try out above.
Purpose
The primary purpose of the script is to get the products of photogrammetry applications into Octane,
although it can be used to populate an Octane Scatter Node with any set of points in a pre existing ascii file.
Photogrammetry applications will create 3D point cloud data from a series of overlapping 2D imates,
other 3d data can then be derived from the point cloud and the photographic images such as textured meshes.
These products can be very useful in creating a context for a subject model.
Description/Instructions
Makes a file suitable for use in an Octane Scatter node from a simple list of xyz locations.
Colored point clouds are supported if RGB(A) color data is present by generating an Instance Color Texture
from these values and assigning a unique instance id to each point.
The user must supply a suitably formatted file as described below. The file will be analyzed and loaded into
the script's UI and the user given the chance to check it. The script will identify lines containing non point
related data and alert the user. The user should check for the presence of a Header at this time and remove
it with a text editor if necessary, or let the script decide which lines belong to a Header and remove them
automatically. The user can select what sort of output they desire and then they can process the file. Processing
will create a .cvs file that can be read by an Octane Scatter Node, and optionally an Instance Color texture that
can be imported into Octane. The three output options are as follows.
Write a space seperated .csv file suitable for input into an Octane scatter node
from an input file that contains only the three cordinates for each point. This format
has utility beyound point clouds and can be used to scatter any object within an
Octane Scene when the original locations are not formatted appropriately for a Scatter Node.
Write a space seperated .csv file suitable for input into an Octane Scatter Node
from an input file that contains only the three cordinates for each point,
appending an instance id to each point record. The assignment of an instance id potentially
makes the data useful for other purposes, such as the assignment of color to points based on
photographic images, or making the scatter data useful for further manipuation by other scripts.
Write a space seperated .csv file suitable for input into an Octane Scatter Node from an input
file that contains the three cordinates for each point and it's rgb(a) color data, appending an
instance id to each point record and creating a seperate Instance Color Texture image to store
the colors. This is the primary method for creating colored point clouds. This option is set as
the default. If this option is selected, a preview of the Instance Color Texture is displayed.
As part of the processing step, the user can optionally have the script create a Node Graph of the all
the nodes required to display the point cloud. The script will create a scatter node and load the newly created
.csv file to locate each point. It will create a Vectron Sphere object, connect a Diffuse Material to it, and
import the Instance Color Texture and connect it to the material so that each point is appropriately colored.
A float value defaulting to 0.01 is also connected to the shere so that its radius can be controlled. The user
should set this to the desired value which will be based on the size of the point cloud's geographic extents,
Larger extents will require larger points. The sphere is connected to the scatter node, and the scatter node is
connected to a placement node. A ground Plane and Geometry Group are also included.
Due to differences in various authoring applications, the whole point cloud might need to be rotated.
Use the placement node to align with Octanes Y axes being up, or instead, try out the 'Y axis up' feature which
attempts to do this by swapping the Y and Z point ordinates and negatively scaling the Z ordinate as it writes the
Scatter Node matrices. The graph must be manually connected to the geometry pin of a Render Target so that it
will appear in the viewport. Remember to adjust the Sphere radius to get the point cloud to display as desired,
they can be difficult to see at times, and have a direct relationship with the Ray epsilon value which must be
smaller than the Radius value. If you can't see your point cloud, check this numeric relationship.
Point Clouds can have large geographic extents, particularly if they contain landforms, and adjusting the
Ray epsilon value might required anyway.
The script has been successfuly used with files in excess of 30 million points, however files of this size
take several minutes (about 3 million points per minute) to process and some time beyond this to return control
of Octane to the user. There is a points thinning capability that will reduce the size of the point cloud by
ignoring every Nth line in the input file.
User Supplied Data.
ASCII files with the following extensions are made accessable via the UI.
*.txt; *.pts; *.xyz; *.pcd *.ply; *.csv
However these files may need to be edited in order to work with the script. Notepad++ is a good free editor
given its ability to manipulate columns of data in an unstructured text file.
The script supports space seperated files without a Header with some combination of the following
x y z r g b a id where x,y,z are 3 dimensional cartesian coordinates, r,g,b (and optionally a) are pixel
channel values between 0 and 255 specifying the points color. The script only supports a certain number of
values per line in the original file. 3 is the minimum containing the X,Y,Z ordinates of a point, 7 is the
maximum containing X,Y,Z and R,G,B,A specifying the red, green, blue, and alpha channel values of each point.
The data must be organised in the correct order i.e. X,Y,Z,R,G,B,A within each line. Some files store additional
data in each line. You can examine the data in the text box that will display the first few lines of the file.
In this way you can check for headers and validate the lines found in the body of the file. The Script has a
feature whereby the user can tell it to ignore certain values\columns within a line.(see discussion of .ply files below).
The script will present the data from the body of the file as illustrated below.
| 1| 2| 3| 4| 5| 6| 7| 8| 9| 10|
To have the script ignore a given column, place an x before the Column Id in the text editor in which it's displayed.
| 1| 2| 3| x4| x5| x6| 7| 8| 9| 10|
This is how you would tell the script to ignore the vector normals contained in the .ply file below. The script will
not let you proceed until the number of remaining columns lies within the range of 3 to 7.
The script will also attempt to remove Header lines prior to processing. If Header lines are detected, it first
backs up the original input file by appending "orig_" to the front of its <filename> becoming <orig_filename>.
it then rewrites the file's body to the original file i.e. <filename> without the Header. For large files this can
take a few minutes, but if you don't delete the new file you can reuse it without having to do this again.
Here is an example of the files produced by the script after a Header is found and the file contains X,Y,Z,R,G,B,A data.
Original filename opened by script: DroneMapper3D.ply
Backup filename with Header in tact: orig_DroneMapper3D.ply
New filename with Header removed: DroneMapper3D.ply
CSV filename with scatter node data: Scatter_DroneMapper3D.cvs
Image Color Texture filename: imgICT_DroneMapper3D.png
Making Point Clouds
You can use third party applications such as those listed below to convert photographs you have taken,
or other point cloud file formats to one of the following *.txt; *.pts; *.xyz; *.pcd *.ply; *.csv supported formats.
Additionall lumalabs, https://lumalabs.ai/ produces a point cloud as a by product of its nerf creation process making
it quite easy to do with a mobile phone.
Autodesk ReCap pro
Agisoft Metashape
Meshroom (free)
Pix4D
3DF Zephyr
Regard3D (free)
PhotoModeler
WebODM (free)
RealityCapture
COLMAP (free)
MeshLab (free)
About Each Supported Format.
.txt
Simple ascii text of no particular content or format. Points stored in .txt files are organized however the
original author sees fit. To be of use to this script, expect one point definition per line, x,y,z being
the first of three substrings in that line, if six or seven substrings are present the next three
or four need to be r,g,b or r,g,b,a values specifying red, green, blue and alpha channels of the point's color.
This script can handle all these conditions as illustrated by the sample lines below. If your .txt file
contains point data organized differently, you can reorganize it using a tool such as the column editor in
Notepad++. Headers are not supported in .txt files and need to be removed, all lines of data need to be numeric.
e.g. three substrings specifying the spatial location of the point.
2.1623 -1.8567 0.7161
1.6836 0.1857 1.6998
2.1676 -1.8555 0.7122
e.g. six substrings specifying the spatial location of the point & r,g,b color values.
2.1623 -1.8567 0.7161 250 248 252
1.6836 0.1857 1.6998 50 41 54
2.1676 -1.8555 0.7122 251 248 253
e.g. seven substrings specifying the spatial location of the point & r,g,b color values + an alpha value
If an alpha value is detected the actual value will be set to 255, i.e. solid during processing.
2.1623 -1.8567 0.7161 250 248 252 252
1.6836 0.1857 1.6998 50 41 54 60
2.1676 -1.8555 0.7122 251 248 253 252
.xyz & .pts
Similar comments to those for .txt files although the extension is more descriptive of what the data represents.
As always, watch out for Header information at the top of the file. The .xyz file extension is used by Matterport
in their MatterPak package and contain six substrings per line as explained above. They work well with this script,
thus you can easily build a point cloud from a Matterport scan.
.pcd (point cloud data)
Comes in binary and ascii versions, the script can only use the ascii version however some of the third party
applications mentioned could be used to convert between the two. This file format has a relatively strict
Header as follows. The order of Header entries is important. Again the Header needs to be removed prior to
processing by the script. The FIELDS definition will tell you if color data is present. See Wikipedia for more info.
# .PCD v.7 - Point Cloud Data file format
VERSION .7
FIELDS x y z rgb
SIZE 4 4 4 4
TYPE F F F F
COUNT 1 1 1 1
WIDTH 213
HEIGHT 1
VIEWPOINT 0 0 0 1 0 0 0
POINTS 213
DATA ascii
0.93773 0.33763 0 4.2108e+06
0.90805 0.35641 0 4.2108e+06
0.81915 0.32 0 4.2108e+06
0.97192 0.278 0 4.2108e+06
.ply
The .ply file format is designed to store more than just a point cloud. It is more akin to the .obj format
than anything discussed so far. Thus point clouds are a special case and you should validate the contents of
the .ply files you have before attempting to use them. They come in binary and ascii versions, the script will
only work with the ascii version. If you have a binary version you need to convert it with a third party app.
Below is a sample of the first few lines of a .ply file containing a point cloud. It contains point location
and rgba data that we can use, however there are three additional substrings in each line, defined in the Header
as floating point numbers, nx,ny,nz all of which are less than 1. The script cannot process these and currently
they need to be removed using an ascii text editor such as Notepad++, or using the built in Column Selection feature.
ply
format ascii 1.0
comment VCGLIB generated
element vertex 2026712
property float x
property float y
property float z
property float nx
property float ny
property float nz
property uchar red
property uchar green
property uchar blue
property uchar alpha
element face 0
property list uchar int vertex_indices
end_header
-177714.5 -929289.9 2028.838 0.03822402 -0.003634897 0.09233478 84 75 73 255
-177715.1 -929291.8 2024.964 -0.02729818 0.01025391 0.09565389 18 17 17 255
-177715.3 -929291.6 2024.841 -0.07316666 0.004162515 0.06803906 19 18 18 255
User Interface
All UI controls have tool tips. The script is heavily commented and well populated with print statements to aid in debugging.
It should be relatively easy to modify to your specific purposes and run using the Octane Script Editor which will show you whats
going on under the hood. There are several variables that can be customized by the user to alter the scripts start up configuration,
they are:
Variable name: Default value:
------------------------------- --------------
local flgFormExpanded = true
local numGetHeaderLines = 20
local numDefaultBitMapDimension = 300
local flgCommaDelimited = true
local flgIgnoreAlpha = true
local numStepSize = 1
local flgYup = false
local numPointRadius = 0.035
local numCubeSide = 0.025
Author Mark Bassett
Last modified 8/06/2023
version 1.00
shortcut ctrl + p
let me know about any bugs etc
best
Mark
