6. Advanced Data Analysis 1: General¶
Spectronon’s data visualization and analysis tools are presented in this section, following the structure of the main menu.
6.1. File Menu¶
Open an ENVI® compatible datacube file. The datacube is added to the resource tree and an image is generated in the image panel.
Hint
Dragging a .bil, .bip or .bsq datacube file into Spectronon will open it.
Opens a saved spectrum file.
Hint
Dragging a .spec spectrum file into Spectronon will open it.
Opens a window that allows you to set a variety of preference options. Options associated with data visualization can be found under the Plotters and workbench tabs at the top of the window.
The Plotters tab opens a window that allows you to control the presentation of the plots presented in the spectral plotter. Additionally, this window also allows you to set the parameters for exporting plot data so it can be manipulated or plotted using other software tools.
The workbench tab opens a window that allows you to set the default image preference. One of the most useful settings is RGB, which generates a Red-Green-Blue image based on the values of three chosen hyperspectral bands, which you can choose in the tool control panel. Selecting the True Color button in the tool control panel produces an image that approximates the colors you would see looking at the object. Adjusting the sliders allows you to generate false-color images. Be sure to click Update after adjusting the sliders. Greyscale is another useful option, which presents a greyscale (black and white) image based on a single hyperspectral band. Again, you can adjust this band using sliders in the tool control panel. Be sure to click Update after moving a slider.
Hint
Check the Auto Update box in the tool control panel so you do not have to continually click Update.
Spectronon enables you to run scripts to best suit your application. A variety of scripts are built in to Spectronon.
You may also run Python scripts you write. opens a window that allows you to browse to your script.
For information on Batch Processing your data go to Section 9: Batch Processing.
The mosaic tool is designed for an airborne application and requires georegistered data to operate. This allows the user to create one image by merging several individual images from a raster dataset.
When making changes to user-written plugins, clicking this will make Spectronon aware of the changes.
Closes Spectronon.
6.2. Datacube Menu¶
6.2.1. New Image¶
Allows you to create a new image in the Image panel. The items in this submenu are implemented as plugins. See Section 11.2: Render Plugins for full documentation.
The most generally useful options are RGB, Greyscale and Stack.
Hint
If the menu is greyed-out, make sure a datacube (not an image) is selected in the resource tree.
Generates a color image based on three bands of your hyperspectral data. These bands can be chosen and adjusted using sliders in the tool control panel. See Section 11.2.7: RGB.
This produces a single-band image from a single hyperspectral band that can be chosen using a slider in the tool control panel. See Section 11.2.4: Greyscale.
This tool enables you to overlay multiple images. This tool is particularly useful for presenting classification results. For example, we may have a True Color RGB image of an object, such as shown below, as well as a classification map of one of the candy types. Using the Stack tool, we can combine these images to show the classification map on top of the RGB True Color image.
The stack tool opens a Stack tab in the tool control panel. Use the slider to select how many images you wish to combine (Stack height). Then click Update. Pull-down menus appear that allow you to select the images you wish to combine. A slider (Alpha) is available for each image that allows you to set its transparency in the combined image. Once set, click Update and your combined image will appear.
6.2.2. New Cube¶
allows the user to create a new datacube from an existing datacube. The items in this submenu are implemented as plugins. See Built-in Plugins: Cube Plugins for full documentation.
6.2.2.1. Utilities¶
Allows you to generate a new, modified datacube from the currently open datacube. A few of the more commonly used utilities are explained below. See Section 11.1.1: Utilities for full documentation.
Generates a new cube by binning spectral and/or spatial channels, which is often useful to either reduce the size of the datacube or improve the signal-to-noise ratio. This choice creates a new image in the Images Window and also generates a Bin Cube tab in the tool control panel with 3 sliders. The Sample Bin and Line Bin allow you to bin pixels along a spatial (x,y) axis. The Sample axis refers to the cross-track axis of the imaging spectrometer, and the Line axis refers to the along-track axis of the imaging spectrometer. The Spectral Bin slider allows you to bin spectral channels, and is likely the most useful of the binning options. After clicking Update, a new binned datacube is generated.
Note
This is a summation, not an average. The datatype of the returned cube may be promoted as required. If you choose Float Mode floating-point data will be returned.
See Section 11.1.1.5: Bin Channels.
Generates an image in the Image panel and a new tab with sliders in the tool control panel that allow you to crop wavelength bands by choosing a new minimum and maximum wavelength within the cube. Once chosen, click the Update button.
Hint
You may want to click on the RGB tab in the tool control panel to reset the bands used to create the image after cropping wavelengths.
See Section 11.1.1.9: Crop Wavelengths.
Generates a new cube by subtracting a background spectrum from all pixels in the datacube. This option is useful, for example, if you are monitoring fluorescent dyes and wish to subtract the background fluorescence of the substrate. See Section 11.1.1.19: Subtract Spectrum.
6.2.2.2. Classify¶
Allows you to generate classification maps of different objects within your hyperspectral data. More information on hyperspectral classification can be found in Section 7: Advanced Data Analysis 2: Hyperspectral Classification. Also see Section 11.1.2: Classify for full classification plugin documentation.
Some of the classification algorithms (SAM, Euclidean Distance, Spectral Unmix) use Spectrum objects as inputs, while the statistics-based classifiers (Logistical Regression, Quadratic Discrimient, Support Vector) require datacubes of the classification target. These are made by selecting the ROI with the lasso or marquee tool, then selecting .
When you select one of the classification options, (e.g. SAM), a new cube will
be generated in the datatree panel. In many plugins, you will need to specify how many Layers you wish to classify (i.e., how
many materials you wish to classify) – you will then need to select a Spectrum or Cube
for each of these Layers in a pull-down menu. The Spectra and Cubes available in the
pull-down menu can be generated either by using one of the ROI tools, marquee
or lasso (
or 
) or by loading a previously saved
spectrum or cube.
Note
a detailed discussion of hyperspectral data classification is beyond the scope of this document. Spectronon provides some of the more commonly used algorithms. For more advanced algorithm capability, please see other packages such as ENVI®. Additionally, custom algorithms can be utilized with Spectronon using the custom plugin capability described in the Spectronon custom plugin manual.
The binary encoding classification technique encodes the data and endmember spectra into zeros and ones, based on whether a band falls below or above the spectrum mean, respectively. See Section 11.1.2.2: Binary Encoding.
This is a commonly used classification algorithm for hyperspectral data that is more sensitive to pixel brightness than SAM. See Section 11.1.2.3: Euclidean Distance.
The SAM classification routine is described in detail in Section 7.2: Spectral Angle Mapper (SAM) Classification. Also see Section 11.1.2.10: Spectral Angle Mapper.
Spectral unmixing deconvolves the signal in each pixel into a linear combination of known spectra. See Section 11.1.2.11: Spectral Unmix.
6.2.2.3. Analyze¶
allows you to perform useful operations on hyperspectral data that will generate new datacubes based on applying analytical functions to your datacube.A few of these are documented here. See Section 11.1.3: Analyze for full documentation.
This tool generates a
new datacube with the first derivative of the spectral curve for each pixel. A
new image of the first derivative datacube is presented in the Image panel. To
see the spectral derivatives, utilize the inspector tool 
, or
use one of the ROI tools marquee or lasso ( or ). The
first derivative curves appear in the spectral plotter.
See Section 11.1.3.3: First Derivative.
A detailed discussion of PCA is beyond the scope of this document (see, for example,
Wikipedia for a discussion). This tool generates a new datacube and image with
the principal component values for each pixel. Additionally, a PCA tab appears
in the tool control panel with a slider that allows you to select the number of
Bands, or PCA components. To see the PCA component magnitudes, utilize the
inspector tool , or use one of the ROI tools marquee or lasso.
( or ). The PCA magnitude curves appear in the
spectral plotter. As with standard hyperspectral data, classification
algorithms can be performed on the new PCA datacube.
See Section 11.1.3.5: PCA.
6.2.2.4. Correct¶
Allows you to correct / calibrate a datacube from a measured reference. This tool also allows you to convert your data to either radiance or reflectance values, as described below. See Section 11.1.4: Correct for full documentation.
Converts raw data to units of radiance (microflicks) based on the instrument’s radiometric calibration file (.icp). See Section 11.1.4.4: Radiance From Raw Data.
Converts radiance data to reflectivity based on a downwelling irradiance spectrum collected with a single point spectrometer and cosine corrector. See Section 11.1.4.6: Reflectance from Radiance Data and Downwelling Irradiance Spectrum.
This approach to converting data to reflectance assumes the input datacube has been corrected spatially, typically by converting to radiance but other approaches would be acceptable. A spectrum of measured reference material is used to correct the datacube spectrally. See Section 11.1.4.7: Reflectance from Radiance Data and Measured Reference Spectrum.
This approach assumes the input datacube has been corrected spatially, typically by converting to radiance but other approaches would be acceptable. A spectrum of spectrally flat reference material (typically Spectralon ™ or Fluorilon ™) is used to correct the datacube spectrally. See Section 11.1.4.8: Reflectance from Radiance Data and Spectrally Flat Reference Spectrum.
Converts raw data to reflectivity based on a downwelling irradiance spectrum collected with a single point spectrometer and cosine corrector. See Section 11.1.4.9: Reflectance from Raw Data and Downwelling Irradiance Spectrum.
Convert raw data to reflectance based on a input datacube of measured reference material. See Section 11.1.4.10: Reflectance from Raw Data and Measured Reference Cube.
Converts raw data to reflectivity based on another datacube of a spectrally flat reference material. See Section 11.1.4.11: Reflectance from Raw Data and Spectrally Flat Reference Cube.
6.2.2.5. Anomaly¶
Anomaly detection tries to determine anomalous outliers in a datacube that do not conform to the expected spectra. This is usually based on a datacube of ‘clutter’, which contains the expected spectra.
See Section 11.1.5: Anomaly for full documentation.
6.2.2.6. Color¶
Allows you to transform your hyperspectral data into CIE
colorspace, providing XYZ, xyY, and LAB values for each pixel. Additionally,
you can determine the 
values for each pixel as compared to a standard set by
the user.
See Section 11.1.6: Color for full documentation.
6.2.2.7. Agricultural¶
Create maps of Hyperspectral Vegetation Indices (HVI’s). HVI’s in Spectronon are discussed in greater depth in Section 8. Also see Section 11.1.7: Agricultural.
6.2.2.8. Clustering¶
Clustering tools are unsupervised classification routines that do not require any a priori information of a spectral image. They group the scene into clusters based on spectral similarity. Supervised classification routines can almost always produce better results, if scene knowledge available. See Built-in Plugins: Cube Plugins - Clustering for full documentation.
6.3. Image Menu¶
Tools for saving or closing images generated from a datacube.
Exports an image as TIFF, PNG, BMP, JPG, or GIF.
Exports a georectified image in Keyhole Markup Language (KML) format.
Exports a georectified image as a TIFF file.
Copies an image to the Windows clipboard.
Closes the selected image.
6.3.1. Filter¶
Changes the filter applied to a generated image. Filters alter the image presentation in some way. Spectronon usually chooses the most appropriate filter automatically. Most commonly this is the contrast filter.
All filter menu items are implemented as plugins. See Section 11.3: Filter Plugins for full documentation.
6.4. Spectrum Menu¶
The Spectrum menu provides tools for manipulating individual spectra.
Hint
If the Spectrum menu is greyed out, verify that a spectrum is selected in the resource tree.
Saves the spectrum selected in the Resource Tree.
Saves a spectrum in a file name of your choice.
Saves a spectrum as a text file.
Renames a spectrum.
Computes the area under the spectral plot for a given spectrum.
Converts a spectrum to irradiance. A downwelling calibration pack is required.
Closes the spectrum selected in the Resource Tree and removes the associated plot from the spectral plotter.
Plots the spectral curve of the selected spectrum in the spectral plotter.
Hides the spectral curve plot in the spectral plotter.
Shows the standard deviation envelope in the spectral plotter.
Hides the standard deviation envelope in the spectral plotter.
Changes the color of the selected spectrum’s spectral curve and generated classification maps.
Highlights the datacube region used to generate the selected spectrum.
Copies a spectrum’s spectral data to the Windows clipboard.
Select a spectrum, then select a region in the spectra window, and then click . The spectral plot will be cropped to the chosen range.
6.5. Selection Menu¶
The menu items provide a number of options for use with the
selection tools (, , or 
). For all of the
following options, you must first select a Region of Interest (ROI).
Note
The menu can be accessed from the main menu or by right-clicking in the image after creating an ROI.
Note
Some of these menu items are implemented as selection plugins. See Section 11.4: Select Plugins for references.
Selects the entire dataset.
Clears the current selection.
Creates a new datacube from the current selection.
This tool will calculate and plot the mean spectrum for the pixels selected in your ROI. See Section 11.4.3: Mean Spectrum.
Show a dialog box with the mean of each band in the selected region. See Section 11.4.4: Show Mean of Selection.
Create and plot the mean profile of the selected region in the Z (band) dimension. If wavelength metadata is present this is equivalent to the Mean Spectrum tool. See Section 11.4.5: Mean Z profile.
Plot the mean first derivative (discrete difference) of the selected area. See Section 11.4.6: Mean First Derivative.
Creates a heatmap showing band-to-band correlations using the Pearson coefficient. See Section 11.4.7: Spectral Correlation Matrix.
A correlation coefficient aligns reflectance data created with downwelling irradiance data to a ground truth target such as a tarp or reflectance standard. Once created, these coefficients are used in the Reflectance Conversion with Downwelling Irradiance plugins to improve the accuracy of the result. See Section 11.4.8: Create Correlation Coefficients.
Uses spectral information to determine the closest RAL Classic color to selected region. Wavelength and reflectance scale factor metadata must be present. Useful for color matching. See Section 11.4.9: Find Closest RAL Classic Color.
Plot the median spectrum within selected area. See Section 11.4.10: Median Spectrum.
Compute and plot the mean Nth derivative (discrete difference) of the selected region. See Section 11.4.11: Mean Nth Derivative.
Randomly selects pixels within the region of interest and exports their spectra (or bands) to a text file. See Section 11.4.12: Save random spectra to text file.
6.5.1. Send to Clipboard¶
This tool allows you to transfer the data within your selected ROI. The first option is , which allows you to paste the mean spectrum values into other applications such as Notepad or Excel. The second option, copies the spectral curves of all the pixels within your selected ROI. This tool is useful for those who want to perform specific statistical analysis on the data from small regions.
Copies mean spectrum as text to clipboard for pasting into other applications. See Section 11.4.13.1: Copy Mean as Text.
Copies all spectra in region as text to clipboard for pasting into other applications. See Section 11.4.13.3: All Spectra as Text.
6.6. Plots Menu¶
The menu items provide tools for the data shown in the plot panel.
Set X-axis and the Y-axis ranges.
Save a plot to an image file.
Save plot data values as a .txt file.
Clears the spectral plot.
Copy plot data to the Windows clipboard as text.
6.7. Datacubes and Header Files¶
Datacubes collected with Resonon imaging spectrometers require two files, one that contains the hyperspectral data, and also a header file that contains important information about the data. Hyperspectral data can be accessed with many tools, for example, Matlab and ENVI, and is compatible with a variety of languages including: Python, C++, Fortran.
6.7.1. Hyperspectral data files¶
ENVI-compatible hyperspectral data can be stored in the following three formats:
Band-Sequential In Band-Sequential (BSQ) format each line of the data is followed by the next line in the same spectral band. Hyperspectral data stored in BSQ format will end with .bsq.
Band-Interleaved-by-Line Data saved in Band-Interleaved-by-Line (BIL) format have the first line of the first band followed by the first line of the second band, followed by the first line of the third band, and so forth. This pattern is repeated for subsequent lines. Hyperspectral data stored in BIL format will end with .bil.
Band-Interleaved-by-Pixel Band-Interleaved-by-Pixel (BIP) data have the first pixel for all bands in sequential order, followed by the second pixel for all bands, and so forth for all pixels. Hyperspectral data stored in BIP format will end with .bip.
Resonon imaging spectrometers will save hyperspectral data in BSQ, BIL, or BIP formats, and Spectronon software will open hyperspectral data in all of these formats.
6.7.2. Hyperspectral data header files¶
Header files for hyperspectral data have the same name as the BSQ, BIL, or BIP files followed by .hdr. The following discussion provides an overview of header files.
Header files can be opened and edited using WordPad. The following list describes some, but not all, of header file commands. Bold indicates the command, quantities in (brackets) are acceptable arguments for the commands, and statements in [square brackets] are additional explanation. Note that the order of these commands can be different from file to file.
ENVI [All ENVI-compatible header files begin with this]
interleave = (bil, bip, bsq) [Tells the software the hyperspectral data format]
data type = (4, 12) [4 is 4-byte floating point; 12 is 2-byte unsigned ; other data types are also supported, but these are common]
lines = (###) [The number of lines in the hyperspectral data where ### is a number]
samples = (###) [The number of samples in the hyperspectral data where ### is a number. This is typically the number of cross-track spatial pixels of the hyperspectral imager.]
bands = (###) [The number of bands in the hyperspectral data where ### is a number]
bit depth = (12, 14) [The bit depth of the hyperspectral data]
shutter = (###) [The shutter time in units of msec where ### is a number. This is mostly for reference, but it may be used by certain plugins]
gain = (#.#) [The camera gain in dB where #.# a number. This is mostly for reference, but it may be used by certain plugins]
framerate = (####) [The framerate at which the data were recorded in frames/sec where ### is a number. This is mostly for reference, but it may be used by certain plugins]
reflectance scale factor = (4095, 1023, 1) [If you divide the data by this factor, the data will scale from 0 to 1, typically for reflectance data]
byte order = (0) [byte order is the ordering of byte significance (http://en.wikipedia.org/wiki/Endianness) Data for Spectronon are LSF or Little Endian which means the least significant bytes come first. Spectronon will not properly open BSF or Big Endian files]
header offset = (0) [This is the number of bytes at the beginning of the binary file that should be ignored].
wavelengths = (###, ###, ###, … ) [List of the spectral band centers in units of nm]
rotation = ((#,#), (#,#), (#,#), (#,#)) [This is a default orientation of the view of a datacube in Spectronon. This is a Spectronon only header option].
label = (name) [The label is used by Spectronon to give a human readable alternative to the file name of a file. If omitted, Spectronon will use the file name
timestamp = (day & time of datacube) [If available, this is when the datacube was recorded. The timestamp is often found on airborne datacubes, and datacubes where a script entered this information explicitly into the header]
The following are for spec.hdr files only, headers for spectrum. Spectrum files are ENVI compatible because they are 1x1 datacubes, but Spectronon adds in these data for its own use.
original cube file = (name of original cube) [This is a Spectronon-only header item for recording the history of a generated spectrum. In the case of a derived datacube, you will see a “history” header value that shows the way this cube was generated]
pointlist = (long list of values) [This is the set of points from the original cube that were averaged together to make this spectrum if the spectrum is a mean]
label color = (#FF00FF) [This is the color used to plot the spectrum]