7. Advanced Data Analysis 1: General

Spectronon’s data visualization and analysis tools are presented in this section, following the structure of the main menu.

7.1. File Menu

_images/File_dropdown6_1-7_1_1.png

File ‣ Open Datacube…

Open an ENVI® compatible datacube file. The datacube is added to the resource tree and an image is generated in the image panel.

Hint

Dragging a .bil, .bip or .bsq datacube file into Spectronon will open it.

File ‣ Open Spectrum…

Opens a saved spectrum file.

Hint

Dragging a .spec spectrum file into Spectronon will open it.

File ‣ Preferences…

Opens a window that allows you to set a variety of preference options. Options associated with data visualization can be found under the Plotters and workbench tabs at the top of the window.

The Plotters tab opens a window that allows you to control the presentation of the plots presented in the spectral plotter. Additionally, this window also allows you to set the parameters for exporting plot data so it can be manipulated or plotted using other software tools.

The workbench tab opens a window that allows you to set the default image preference. One of the most useful settings is RGB, which generates a Red-Green-Blue image based on the values of three chosen hyperspectral bands, which you can choose in the tool control panel. Selecting the True Color button in the tool control panel produces an image that approximates the colors you would see looking at the object. Adjusting the sliders allows you to generate false-color images. Be sure to click Update after adjusting the sliders. Greyscale is another useful option, which presents a greyscale (black and white) image based on a single hyperspectral band. Again, you can adjust this band using sliders in the tool control panel. Be sure to click Update after moving a slider.

Hint

Check the Auto Update box in the tool control panel so you do not have to continually click Update.

File ‣ Built in Scripts

Spectronon enables you to run scripts to best suit your application. A variety of scripts are built in to Spectronon.

File ‣ Run Script…

You may also run Python scripts you write. File ‣ Run Script opens a window that allows you to browse to your script.

File ‣ Batch Processor…

For information on Batch Processing your data go to Section 10: Batch Processing.

File ‣ Mosaic Tool…

The mosaic tool is designed for an airborne application and requires georegistered data to operate. This allows the user to create one image by merging several individual images from a raster dataset.

File ‣ Reload Plugins

When making changes to user-written plugins, clicking this will make Spectronon aware of the changes.

File ‣ Exit

Closes Spectronon.

7.2. Datacube Menu

7.2.1. New Image

Allows you to create a new image in the Image panel. The items in this submenu are implemented as plugins. See Section 12.2: Render Plugins for full documentation.

The most generally useful options are RGB, Greyscale and Stack.

_images/GreyscaleImageMenu.png

Hint

If the New Image menu is greyed-out, make sure a datacube (not an image) is selected in the resource tree.

Datacube ‣ New Image ‣ RGB

Generates a color image based on three bands of your hyperspectral data. These bands can be chosen and adjusted using sliders in the tool control panel. See Section 12.2.7: RGB.

Datacube ‣ New Image ‣ Greyscale

This produces a single-band image from a single hyperspectral band that can be chosen using a slider in the tool control panel. See Section 12.2.4: Greyscale.

Datacube ‣ New Image ‣ Stack

This tool enables you to overlay multiple images. This tool is particularly useful for presenting classification results. For example, we may have a True Color RGB image of an object, such as shown below, as well as a classification map of one of the candy types. Using the Stack tool, we can combine these images to show the classification map on top of the RGB True Color image.

_images/Stack_C4.png

The stack tool opens a Stack tab in the tool control panel. Use the slider to select how many images you wish to combine (Stack height). Then click Update. Pull-down menus appear that allow you to select the images you wish to combine. A slider (Alpha) is available for each image that allows you to set its transparency in the combined image. Once set, click Update and your combined image will appear.

See Section 12.2.10: Stack.

7.2.2. New Cube

Datacube -> New Cube allows the user to create a new datacube from an existing datacube. The items in this submenu are implemented as plugins. See Built-in Plugins: Cube Plugins for full documentation.

7.2.2.1. Utilities

Allows you to generate a new, modified datacube from the currently open datacube. A few of the more commonly used utilities are explained below. See Section 12.1.1: Utilities for full documentation.

_images/NewCube_Utilities_Menu.png

Datacube ‣ New Cube ‣ Utilities ‣ Bin Channels

Generates a new cube by binning spectral and/or spatial channels, which is often useful to either reduce the size of the datacube or improve the signal-to-noise ratio. This choice creates a new image in the Images Window and also generates a Bin Cube tab in the tool control panel with 3 sliders. The Sample Bin and Line Bin allow you to bin pixels along a spatial (x,y) axis. The Sample axis refers to the cross-track axis of the imaging spectrometer, and the Line axis refers to the along-track axis of the imaging spectrometer. The Spectral Bin slider allows you to bin spectral channels, and is likely the most useful of the binning options. After clicking Update, a new binned datacube is generated.

Note

This is a summation, not an average. The datatype of the returned cube may be promoted as required. If you choose Float Mode floating-point data will be returned.

See Section 12.1.1.4: Bin Channels.

Datacube ‣ New Cube ‣ Utilities ‣ Crop Wavelengths

Generates an image in the Image panel and a new tab with sliders in the tool control panel that allow you to crop wavelength bands by choosing a new minimum and maximum wavelength within the cube. Once chosen, click the Update button.

Hint

You may want to click on the RGB tab in the tool control panel to reset the bands used to create the image after cropping wavelengths.

See Section 12.1.1.8: Crop Wavelengths.

Datacube ‣ New Cube ‣ Utilities ‣ Subtract Spectrum

Generates a new cube by subtracting a background spectrum from all pixels in the datacube. This option is useful, for example, if you are monitoring fluorescent dyes and wish to subtract the background fluorescence of the substrate. See Section 12.1.1.17: Subtract Spectrum.

7.2.2.2. Classify

Allows you to generate classification maps of different objects within your hyperspectral data. More information on hyperspectral classification can be found in Section 8: Advanced Data Analysis 2: Hyperspectral Classification. Also see Section 12.1.2: Classify for full classification plugin documentation.

_images/NewCube_Classify_SAM_Menu.png

Some of the classification algorithms (SAM, Euclidean Distance, Spectral Unmix) use Spectrum objects as inputs, while the statistics-based classifiers (Logistical Regression, Quadratic Discrimient, Support Vector) require datacubes of the classification target. These are made by selecting the ROI with the lasso or marquee tool, then selecting Create Cube from Selection.

When you select one of the classification options, (e.g. SAM), a new cube will be generated in the datatree panel. In many plugins, you will need to specify how many Layers you wish to classify (i.e., how many materials you wish to classify) – you will then need to select a Spectrum or Cube for each of these Layers in a pull-down menu. The Spectra and Cubes available in the pull-down menu can be generated either by using one of the ROI tools, marquee or lasso (marquee tool or lasso tool) or by loading a previously saved spectrum or cube.

Note

a detailed discussion of hyperspectral data classification is beyond the scope of this document. Spectronon provides some of the more commonly used algorithms. For more advanced algorithm capability, please see other packages such as ENVI®. Additionally, custom algorithms can be utilized with Spectronon using the custom plugin capability described in the Spectronon custom plugin manual.

Datacube ‣ New Cube ‣ Classify ‣ Binary Encoding Classification

The binary encoding classification technique encodes the data and endmember spectra into zeros and ones, based on whether a band falls below or above the spectrum mean, respectively. See Section 12.1.2.2: Binary Encoding.

Datacube ‣ New Cube ‣ Classify ‣ Euclidian Distance

This is a commonly used classification algorithm for hyperspectral data that is more sensitive to pixel brightness than SAM. See Section 12.1.2.3: Euclidean Distance.

Datacube ‣ New Cube ‣ Classify ‣ Spectral Angle Mapper (SAM)

The SAM classification routine is described in detail in Section 8.2: Spectral Angle Mapper (SAM) Classification. Also see Section 12.1.2.9: Spectral Angle Mapper.

Datacube ‣ New Cube ‣ Classify ‣ Spectral Unmix

Spectral unmixing deconvolves the signal in each pixel into a linear combination of known spectra. See Section 12.1.2.10: Spectral Unmix.

7.2.2.3. Analyze

Datacube ‣ New Cube ‣ Analyze allows you to perform useful operations on hyperspectral data that will generate new datacubes based on applying analytical functions to your datacube.A few of these are documented here. See Section 12.1.3: Analyze for full documentation.

_images/NewCube_Analyze_FirstDerivative_Menu.png

Datacube ‣ New Cube ‣ Analyze ‣ First Derivative

This tool generates a new datacube with the first derivative of the spectral curve for each pixel. A new image of the first derivative datacube is presented in the Image panel. To see the spectral derivatives, utilize the inspector tool inspector tool, or use one of the ROI tools marquee or lasso (marquee tool or lasso tool). The first derivative curves appear in the spectral plotter. See Section 12.1.3.3: First Derivative.

Datacube ‣ New Cube ‣ Analyze ‣ Principal Component Analysis (PCA)

A detailed discussion of PCA is beyond the scope of this document (see, for example, Wikipedia for a discussion). This tool generates a new datacube and image with the principal component values for each pixel. Additionally, a PCA tab appears in the tool control panel with a slider that allows you to select the number of Bands, or PCA components. To see the PCA component magnitudes, utilize the inspector tool inspector tool, or use one of the ROI tools marquee or lasso. (marquee tool or lasso tool). The PCA magnitude curves appear in the spectral plotter. As with standard hyperspectral data, classification algorithms can be performed on the new PCA datacube. See Section 12.1.3.5: PCA.

7.2.2.4. Correct

Allows you to correct / calibrate a datacube from a measured reference. This tool also allows you to convert your data to either radiance or reflectance values, as described below. See Section 12.1.4: Correct for full documentation.

_images/NewCube_Correct_Menu.png

Datacube ‣ New Cube ‣ Correct ‣ Radiance From Raw Data

Converts raw data to units of radiance (microflicks) based on the instrument’s radiometric calibration file (.icp). See Section 12.1.4.4: Radiance From Raw Data.

Datacube ‣ New Cube ‣ Correct ‣ Reflectance from Radiance Data and Downwelling Irradiance Spectrum

Converts radiance data to reflectivity based on a downwelling irradiance spectrum collected with a single point spectrometer and cosine corrector. See Section 12.1.4.5: Reflectance from Radiance Data and Downwelling Irradiance Spectrum.

Datacube ‣ New Cube ‣ Correct ‣ Reflectance from Radiance Data and Measured Reference Spectrum

This approach to converting data to reflectance assumes the input datacube has been corrected spatially, typically by converting to radiance but other approaches would be acceptable. A spectrum of measured reference material is used to correct the datacube spectrally. See Section 12.1.4.6: Reflectance from Radiance Data and Measured Reference Spectrum.

Datacube ‣ New Cube ‣ Correct ‣ Reflectance from Radiance Data and Spectrally Flat Reference Spectrum

This approach assumes the input datacube has been corrected spatially, typically by converting to radiance but other approaches would be acceptable. A spectrum of spectrally flat reference material (typically Spectralon ™ or Fluorilon ™) is used to correct the datacube spectrally. See Section 12.1.4.7: Reflectance from Radiance Data and Spectrally Flat Reference Spectrum.

Datacube ‣ New Cube ‣ Correct ‣ Reflectance from Raw Data and Downwelling Irradiance Spectrum

Converts raw data to reflectivity based on a downwelling irradiance spectrum collected with a single point spectrometer and cosine corrector. See Section 12.1.4.8: Reflectance from Raw Data and Downwelling Irradiance Spectrum.

Datacube ‣ New Cube ‣ Correct ‣ Reflectance from Raw Data and Measured Reference Cube

Convert raw data to reflectance based on a input datacube of measured reference material. See Section 12.1.4.9: Reflectance from Raw Data and Measured Reference Cube.

Datacube ‣ New Cube ‣ Correct ‣ Reflectance from Raw Data and Spectrally Flat Reference Cube

Converts raw data to reflectivity based on another datacube of a spectrally flat reference material. See Section 12.1.4.10: Reflectance from Raw Data and Spectrally Flat Reference Cube.

7.2.2.5. Anomaly

Anomaly detection tries to determine anomalous outliers in a datacube that do not conform to the expected spectra. This is usually based on a datacube of ‘clutter’, which contains the expected spectra.

See Section 12.1.5: Anomaly for full documentation.

7.2.2.6. Color

Allows you to transform your hyperspectral data into CIE colorspace, providing XYZ, xyY, and LAB values for each pixel. Additionally, you can determine the \Delta E values for each pixel as compared to a standard set by the user.

See Section 12.1.6: Color for full documentation.

7.2.2.7. Agricultural

Create maps of Hyperspectral Vegetation Indices (HVI’s). HVI’s in Spectronon are discussed in greater depth in Section 9. Also see Section 12.1.7: Agricultural.

7.2.2.8. Clustering

Clustering tools are unsupervised classification routines that do not require any a priori information of a spectral image. They group the scene into clusters based on spectral similarity. Supervised classification routines can almost always produce better results, if scene knowledge available. See Built-in Plugins: Cube Plugins - Clustering for full documentation.

7.2.3. Datacube Menu Misc.

Datacube ‣ Save Cube

Saves the selected datacube.

Datacube ‣ Save Cube As

Saves the currently open datacube under a new name or location.

Datacube ‣ Rename Cube

Renames the selected datacube.

Datacube ‣ Close Cube

Removes the selected datacube from the Resource Tree.

7.3. Image Menu

Tools for saving or closing images generated from a datacube.

_images/Image_Menu.png

Image ‣ Export Image

Exports an image as TIFF, PNG, BMP, JPG, or GIF.

Export Image as KML

Exports a georectified image in Keyhole Markup Language (KML) format.

Export Image as GeoTiff

Exports a georectified image as a TIFF file.

Copy Image to Clipboard

Copies an image to the Windows clipboard.

Close Image

Closes the selected image.

7.3.1. Filter

Changes the filter applied to a generated image. Filters alter the image presentation in some way. Spectronon usually chooses the most appropriate filter automatically. Most commonly this is the contrast filter.

All filter menu items are implemented as plugins. See Section 12.3: Filter Plugins for full documentation.

7.4. Spectrum Menu

The Spectrum menu provides tools for manipulating individual spectra.

_images/Spectrum_Menu.png

Hint

If the Spectrum menu is greyed out, verify that a spectrum is selected in the resource tree.

Spectrum ‣ Save Spectrum

Saves the spectrum selected in the Resource Tree.

Spectrum ‣ Save Spectrum As

Saves a spectrum in a file name of your choice.

Spectrum ‣ Export Spectrum as Text File

Saves a spectrum as a text file.

Spectrum ‣ Rename Spectrum

Renames a spectrum.

Spectrum ‣ Integrate Spectrum

Computes the area under the spectral plot for a given spectrum.

Spectrum ‣ Convert Spectrum To Irradiance

Converts a spectrum to irradiance. A downwelling calibration pack is required.

Spectrum ‣ Close Spectrum

Closes the spectrum selected in the Resource Tree and removes the associated plot from the spectral plotter.

Spectrum ‣ Show Plot

Plots the spectral curve of the selected spectrum in the spectral plotter.

Spectrum ‣ Hide Plot

Hides the spectral curve plot in the spectral plotter.

Spectrum ‣ Show Standard Deviation

Shows the standard deviation envelope in the spectral plotter.

Spectrum ‣ Hide Standard Deviation

Hides the standard deviation envelope in the spectral plotter.

Spectrum ‣ Set Label Color

Changes the color of the selected spectrum’s spectral curve and generated classification maps.

Spectrum ‣ Show Region

Highlights the datacube region used to generate the selected spectrum.

Spectrum ‣ Copy Spectrum as Text

Copies a spectrum’s spectral data to the Windows clipboard.

Spectrum ‣ Crop Spectrum

Select a spectrum, then select a region in the spectra window, and then click Spectrum ‣ Crop Spectrum. The spectral plot will be cropped to the chosen range.

7.5. Selection Menu

The Selection menu items provide a number of options for use with the selection tools (marquee tool, lasso tool, or floodfill tool). For all of the following options, you must first select a Region of Interest (ROI).

_images/Selection_Menu.png

Note

The Selection menu can be accessed from the main menu or by right-clicking in the image after creating an ROI.

Note

Some of these menu items are implemented as selection plugins. See Section 12.4: Select Plugins for references.

Selection ‣ Select All

Selects the entire dataset.

Selection ‣ Select None

Clears the current selection.

Selection ‣ Create Cube From Selection

Creates a new datacube from the current selection.

Selection ‣ Mean Spectrum

This tool will calculate and plot the mean spectrum for the pixels selected in your ROI. See Section 12.4.2: Mean Spectrum.

Selection ‣ Show Mean of Selection

Show a dialog box with the mean of each band in the selected region. See Section 12.4.3: Show Mean of Selection.

Selection ‣ Mean Z profile

Create and plot the mean profile of the selected region in the Z (band) dimension. If wavelength metadata is present this is equivalent to the Mean Spectrum tool. See Section 12.4.4: Mean Z profile.

Selection ‣ Mean First Derivative

Plot the mean first derivative (discrete difference) of the selected area. See Section 12.4.5: Mean First Derivative.

Selection ‣ Spectral Correlation Matrix

Creates a heatmap showing band-to-band correlations using the Pearson coefficient. See Section 12.4.6: Spectral Correlation Matrix.

Selection ‣ Create Correlation Coefficients

A correlation coefficient aligns reflectance data created with downwelling irradiance data to a ground truth target such as a tarp or reflectance standard. Once created, these coefficients are used in the Reflectance Conversion with Downwelling Irradiance plugins to improve the accuracy of the result. See Section 12.4.7: Create Correlation Coefficients.

Selection ‣ Find Closest RAL Classic Color

Uses spectral information to determine the closest RAL Classic color to selected region. Wavelength and reflectance scale factor metadata must be present. Useful for color matching. See Section 12.4.8: Find Closest RAL Classic Color.

Selection ‣ Median Spectrum

Plot the median spectrum within selected area. See Section 12.4.9: Median Spectrum.

Selection ‣ Mean Nth Derivative

Compute and plot the mean Nth derivative (discrete difference) of the selected region. See Section 12.4.10: Mean Nth Derivative.

Selection ‣ Save random spectra to text file

Randomly selects pixels within the region of interest and exports their spectra (or bands) to a text file. See Section 12.4.11: Save random spectra to text file.

7.5.1. Send to Clipboard

This tool allows you to transfer the data within your selected ROI. The first option is Copy Mean as Text, which allows you to paste the mean spectrum values into other applications such as Notepad or Excel. The second option, All Spectra as Text copies the spectral curves of all the pixels within your selected ROI. This tool is useful for those who want to perform specific statistical analysis on the data from small regions.

Selection ‣ Send to Clipboard ‣ Copy Mean as Text

Copies mean spectrum as text to clipboard for pasting into other applications. See Section 12.4.12.1: Copy Mean as Text.

Selection ‣ Send to Clipboard ‣ All Spectra as Text

Copies all spectra in region as text to clipboard for pasting into other applications. See Section 12.4.12.3: All Spectra as Text.

7.6. Plots Menu

The Plots menu items provide tools for the data shown in the plot panel.

_images/Plots_menu_254.png

Plots ‣ [plot type] ‣ Set Range

Set X-axis and the Y-axis ranges.

Plots ‣ [plot type] ‣ Save as Image

Save a plot to an image file.

Plots ‣ [plot type] ‣ Save as Text

Save plot data values as a .txt file.

Plots ‣ Spectral Plotter ‣ Clear

Clears the spectral plot.

Plots ‣ [plot type] ‣ Copy to Clipboard as Text

Copy plot data to the Windows clipboard as text.

7.7. Datacubes and Header Files

Datacubes collected with Resonon imaging spectrometers require two files, one that contains the hyperspectral data, and also a header file that contains important information about the data. Hyperspectral data can be accessed with many tools, for example, Matlab and ENVI, and is compatible with a variety of languages including: Python, C++, Fortran.

7.7.1. Hyperspectral data files

ENVI-compatible hyperspectral data can be stored in the following three formats:

Band-Sequential In Band-Sequential (BSQ) format each line of the data is followed by the next line in the same spectral band. Hyperspectral data stored in BSQ format will end with .bsq.

Band-Interleaved-by-Line Data saved in Band-Interleaved-by-Line (BIL) format have the first line of the first band followed by the first line of the second band, followed by the first line of the third band, and so forth. This pattern is repeated for subsequent lines. Hyperspectral data stored in BIL format will end with .bil.

Band-Interleaved-by-Pixel Band-Interleaved-by-Pixel (BIP) data have the first pixel for all bands in sequential order, followed by the second pixel for all bands, and so forth for all pixels. Hyperspectral data stored in BIP format will end with .bip.

Resonon imaging spectrometers will save hyperspectral data in BSQ, BIL, or BIP formats, and Spectronon software will open hyperspectral data in all of these formats.

7.7.2. Hyperspectral data header files

Header files for hyperspectral data have the same name as the BSQ, BIL, or BIP files followed by .hdr. The following discussion provides an overview of header files.

Header files can be opened and edited using WordPad. The following list describes some, but not all, of header file commands. Bold indicates the command, quantities in (brackets) are acceptable arguments for the commands, and statements in [square brackets] are additional explanation. Note that the order of these commands can be different from file to file.

ENVI [All ENVI-compatible header files begin with this]

interleave = (bil, bip, bsq) [Tells the software the hyperspectral data format]

data type = (4, 12) [4 is 4-byte floating point; 12 is 2-byte unsigned ; other data types are also supported, but these are common]

lines = (###) [The number of lines in the hyperspectral data where ### is a number]

samples = (###) [The number of samples in the hyperspectral data where ### is a number. This is typically the number of cross-track spatial pixels of the hyperspectral imager.]

bands = (###) [The number of bands in the hyperspectral data where ### is a number]

bit depth = (12, 14) [The bit depth of the hyperspectral data]

shutter = (###) [The shutter time in units of msec where ### is a number. This is mostly for reference, but it may be used by certain plugins]

gain = (#.#) [The camera gain in dB where #.# a number. This is mostly for reference, but it may be used by certain plugins]

framerate = (####) [The framerate at which the data were recorded in frames/sec where ### is a number. This is mostly for reference, but it may be used by certain plugins]

reflectance scale factor = (4095, 1023, 1) [If you divide the data by this factor, the data will scale from 0 to 1, typically for reflectance data]

byte order = (0) [byte order is the ordering of byte significance (http://en.wikipedia.org/wiki/Endianness) Data for Spectronon are LSF or Little Endian which means the least significant bytes come first. Spectronon will not properly open BSF or Big Endian files]

header offset = (0) [This is the number of bytes at the beginning of the binary file that should be ignored].

wavelengths = (###, ###, ###, … ) [List of the spectral band centers in units of nm]

rotation = ((#,#), (#,#), (#,#), (#,#)) [This is a default orientation of the view of a datacube in Spectronon. This is a Spectronon only header option].

label = (name) [The label is used by Spectronon to give a human readable alternative to the file name of a file. If omitted, Spectronon will use the file name

timestamp = (day & time of datacube) [If available, this is when the datacube was recorded. The timestamp is often found on airborne datacubes, and datacubes where a script entered this information explicitly into the header]

The following are for spec.hdr files only, headers for spectrum. Spectrum files are ENVI compatible because they are 1x1 datacubes, but Spectronon adds in these data for its own use.

original cube file = (name of original cube) [This is a Spectronon-only header item for recording the history of a generated spectrum. In the case of a derived datacube, you will see a “history” header value that shows the way this cube was generated]

pointlist = (long list of values) [This is the set of points from the original cube that were averaged together to make this spectrum if the spectrum is a mean]

label color = (#FF00FF) [This is the color used to plot the spectrum]