Plotting geological attitudes in stereonets using QGIS

A new plugin for QGIS, geocouche, allows to plot geological data attitudes stored in stereonets, using the plotting utilities provided by the apsg module by Ondrej Lexa. Data can be stored in geological layers or entered as text, and plotted as great circles or axis. The current release, as of April, 4th,  is vers. 1.0 and  can be downloaded from: https://github.com/mauroalberti/geocouche/releases

How it works? The following description derives from the tool help. 
With the current geocouche  release, it is possible:

1. to calculate the angles between planes stored in a layer and a reference plane
2. to plot data from layers or texts in stereonets

These two tools are available from the QGIS plugin interface.

Fig. 1. geocouche interface.

Geological angles

This tool allows to calculate the angles (as degrees) between a reference plane and the (eventually selected) features in a point layer (Fig. 2).

Fig. 2. Geological angles calculation interface.

It can be applied to determine the degree of misalignement between a reference (for instance, regional) measure and local geological measures.

Definition of parameters

The user has to define the two fields storing the azimuth (dip direction or RHR strike) and the dip angle of each feature, the attitude of the reference plane, and the name of the output shapefile with a new field storing the calculated angle (Fig. 3).

Fig. 3. Definition of parameters for angle calculation.

Geological stereonets

It allows to produce stereonets depicting geological plane and axis attitudes (Fig. 4). There are three steps:

1. Choice of input data
2. Definition of plot style
3. Data plotting

Fig. 4. Stereonet interface.

Choice of input data

It is possible to use input data from data stored in a point layer (Layer tab, Fig. 5) or to use text input (Text tab, Fig. 6).

Input from point layer

When using a point layer (already loaded in the TOC), plane and/or axis attitudes are defined via the fields storing their values (Fig. 5). When a selection is defined, only the selected features will be considered.

Fig. 5. Input from point layer interface.

Input from text

The input can be inserted into a text window (Fig. 6), defining if data consist of:

• planes

• axes

• planes and axes

Fig. 6. Input from text interface.

Another option to take care of, is,  for plane data, whether orientations are expressed using dip direction or RHR strike.

Plot style

Styles can be defined for both great circles and poles: color, width/size, line/marker style, and transparency (Fig. 7). Settings are stored in memory.

Fig. 7. Plot style interface.

Stereonet plotting

Plots can use a new or a pre-existing stereonet (provided it has not been previously closed) (Fig. 8).

Fig. 8. Stereonet plot interface.

Planes can be plotted as great circles or as plane normals, axes as poles or as normal great circles. An example of stereonet is shown in Fig. 9.

Fig. 9. Stereonet example.

A Linux tool for calculating local best-fit plane attitudes from geological traces

The topographic traces of geological surfaces store information on the 3D attitudes of the geological surfaces. Knowing the coordinates of the intersection points between a topographic surface and a geological surface, it is therefore possible to estimate the local attitudes of a geological surface.

 

The tools

 

A short description of geoSurfDEM: it is composed of two tools, IntersectDEM and the new BestFitGeoplanes.
The former allows to calculate the intersection points between a geological surfaces (stored in the VTK format) and a DEM. The latter, that will be described in the current post, allows to estimate the local attitudes given a set of 3D points, all deriving from the intersection between a single geological surface and a topographic surface. BestFitGeoplanes is developed using C++ and Fortran. The algorithm uses Singular Value Decomposition (SVD) in order to invert the local traces into local best-fit planes. 

The input data is constituted by a set of points (their x, y and z coordinates), all related to a single, continuous geological surface. How to process them in order to derive the local attitudes? Since the source points are unconnected points (not lines), all deriving from a single, continuous surface, the 2D space is discretized into a raster, using a user-defined cell size.
Based on the points falling into a cell, we have three possible cases:

1. no points falling into the cell;
2. one or two points falling into the cell;
3. three or more points in the cell.

In the first and second case, no attitude inversion via SVD is possible.
In the third case, when the points are not all collinear, a solution is provided by the SVD method. This solution is attributed to the grid cell.
The algorithm output will therefore consist in a gridded set of points for which the local attitudes have been inverted.
 
What is the difference with deriving the attitude via a spatial interpolation using for instance kriging? These interpolations implicitly assume a 2.5D surface and do not allow 3D surfaces. Geological surfaces, on the other hand, due to folding, can be 3D surfaces, i.e. with more then one point for each x-y position. Local inversion via SVD does not constrain the geological surfaces to be 2.5D,  but that may be locally modeled via a planar surface.

 

Compilation

 

This application is developed in Linux and is available at: https://github.com/mauroalberti/geoSurfDEM.
A makefile is available for compiling this tool in a Linux environment.
The compilation sequence is:
cd path/to/source/files
make
make clean

Lapack and BLAS libraries must be available. The makefile assumes that Lapack is available in usr/lib/lapack (with name lapack) and BLAS in usr/lib/libblas (name blas). Modify the makefile accordingly, to adapt to your settings. Alternatively, an example of commands to build it (always in a Linux environment and with same libraries settings) is in https://github.com/mauroalberti/geoSurfDEM/blob/master/BestFitGeoplanes/compile

 

Use

 

Having compiled the application, it is possible to run it as console application (Fig. 1). The only user interaction after launching the application is providing a parameter file name ("param.txt" in Fig. 1 example).

Fig. 1. Example of a run of the application.

 

The input parameter file is a text file that lists six pieces of information:

1. the path of text file storing the 3D coordinates (x, y and z) of the intersection points of a single, continuous, geological surface;
2. the number of header lines in the file referenced in point 1;
3. the path  of the text file in which the georeferenced results will be stored;
4. the path to the analysis report file;
5. the path of the output grid (in ESRI ASCII grid format) that will list the number of intersection points for each grid cell;
6. the output grid cell size;

 An example is available in https://github.com/mauroalberti/geoSurfDEM/tree/master/BestFitGeoplanes, in the param.txt file:

/home/mauro/Documents/Ricerca/Codice/Geostrutturale/geoSurfDEM/test_data/BestFitGeoplanes/inters_malpi_135_35.csv
1
/home/mauro/Documents/Ricerca/Codice/Geostrutturale/geoSurfDEM/test_data/BestFitGeoplanes/bfg_malpi_13535_100.txt
/home/mauro/Documents/Ricerca/Codice/Geostrutturale/geoSurfDEM/test_data/BestFitGeoplanes/rep_malpi_13535_100.txt
/home/mauro/Documents/Ricerca/Codice/Geostrutturale/geoSurfDEM/test_data/BestFitGeoplanes/pts_malpi_13535_100.asc
100 

The application output is represented by the three files listed in points 3-5 of the previous list. The more important is the inverted result file (point # 3), that consists in a csv file listing a few fields (see Fig. 2):

1. x: x coordinate of the cell grid center;
2. y: y coordinate of the cell grid center;
3. pt_num: number of points used for the inversion. Required minimum is 3.
4. dip_dir: inverted dip direction for geoplane points in cell;
5. dip_ang: inverted dip angle for geoplane points in cell;
6. x_range: spatial range along the x-direction (E-W) for the inverted points in the considered cell;
7. y_range: spatial range along the y-direction (N-S) for the inverted points in the considered cell;
8. z_range: spatial range along the z-direction (vertical) for the inverted points in the considered cell;
9. pseudo-volume: "pseudo"-volume defined by the inverted points in the cell, given by the product of the three previous ranges (i.e., x_range, y_range and z_range) - possibly removed in successive releases.

Fig. 2. Tabular view of the content of the inversion result file, as viewed by importing the file in QGIS.

 

A theoretical case study 

 

This post presents a practical assessment of the result, by using a theoretical surface for which the expected local attitudes are known (currently, as of February 2017, all example data are available at https://github.com/mauroalberti/geoSurfDEM/tree/master/test_data).
A theoretical plane has been generated using simSurf, with dip direction equal to 135° and dip angle of 35°.
This plane, saved in VTK format from within simSurf, has been used as input plane to be intersecated with a natural topographic surface, of the Mt. Alpi zone (Lucania, Southern Italy), using the IntersectDEM of geoSurfDEM.
The resulting intersection points are saved as a csv file, that can be used as input for the BestFitGeoplanes. The intersection points, to be used for the best-fit-plane local inversions, are represented in Fig. 3.

Fig. 3. Input points (yellow) representing the theoretical intersection between a geoplane oriented 135°/35° (dip dir. and dip angle) and a DEM topography (Mt. Alpi zone, Lucania, Southern Italy). Visualization using QGIS.

 

Using BestFitGeoplanes with the previously described param.txt file, we obtain a result that, for the georeferenced point part, imported in QGIS is shown in Fig. 4.

Fig. 4. The same as in Fig. 3, plus superposed the gridded points with inverted results (orange dots). Visualization with QGIS.

 

How much the inverted results conform to the theoretical input source, that as said is a plane with a dip direction of 135° and a dip angle of 35°?
A stereoplot, created with the geocouche plugin for QGIS, illustrates the degree of  concordance between the source attitude (135°/35°, blue great circle in Fig. 5) and the inverted local attitudes (semi-transparent orange great circles in Fig. 5).
The majority of inverted data conform closely to the expected result, while a few inversions show a minor deviation from the expected result.

 

Fig. 5. Stereonet representing the inferred local plane attitudes as semi-opaque orange great circles, and the source geological plane attitude (135°/35°) as blue great circle. Created with geocouche.

The deviations of the inverted results from the expected value were calculated using the "Geological angles" of geocouche and the statistics calculated with QGIS (see Fig. 6). The maximum is 9.3° and the minimum almost zero, while the median and mean deviation values are lower than 0.5°. The standard deviation is about 1.2°. So in general we can be quite confident in the generated results. More in-depth analyses of the deviations of expected-versus-inferred results could be the subject of a still to-be-written paper.

Fig. 6. Statistics for angular deviations of the calculated results from the theoretical test case.

Edits

2022-12-29: improved paragraph styling; modified input parameters description (items 5 and 6)

geoSurfDEM: a C++ console application for determining intersections between 3D geological surfaces and topography

The determination of the theoretical intersections between digital 3D geological surfaces and topography could be of potential help for studying the field attitudes of natural geological surfaces, as mapped from outcrops or from aerial and satellite images. Since geological structures have complex geometries, the analysis of the relationships between 3D surfaces and topography requires tools that can process 3D geological surfaces.

geoSurfDEM aims at determining:

a) the theoretical intersections between a 3D surface and a topography
b) the local 3D attitude of that surface at each intersection point

How does it work?

Below you see the screenshot of an application run in a Linux shell. When compiled for Windows, the procedure is identical. The total run time can be quite long, many minutes or more.

What is to note?

After the application header display, the user is asked for the name of a text file. In this example, "input_files.txt" is provided, the name of a file located in the same directory as the running application. This file provides the paths of three files:

1) DEM, in ESRI ASCII grid format
2) geological surface, in (old) VTK text format
3) output csv file storing for each row: x-y-z-dip direction-dip angle

An example of input text file is the following:

./publ_data/dem_malpi_aster_wgs84utm33.asc
./publ_data/geological_plane.vtk
./publ_data/intersections.csv


Examples of input data files (i.e., DEM ASCII grid, VTK geosurface file, CSV intersection result) are present in the publ_data subdirectory.

Afterwards, the application outputs a few informative messages about the number of found features and at the last prints out the number of found intersecting points, hopefully greater than zero. The results are stored in the text file referenced by the third path in the input text file.

Example of use

To present the application and check the validity of its results, we use a theoretical test case, i.e. a geological plane with a desired attitude 135°/35°, and with a spatial extent fitting that of the test DEM, covering the Mt. Alpi zone (Basilicata, Southern Italy), derived from global ASTER data.
You can export a DEM in ESRI ASCII grid format with Saga GIS (in addition to ArcGIS).

Creation of test geological plane

The geological plane is created and saved as a VTK text file with simSurf. With this Python 2.7 tool, it is possible to simulate geological surfaces by using analytical formulas.

simSurf is subdivide in two modules:

a) geosurface_simulation.py: creates, geolocates and saves/exports an analytical surface
b) geosurface_deformation.py: reads an analytical surface created by the previous module, deforms it and saves/exports.

Horizontal plane creation

So we start creating a horizontal plane with the Geosurface simulation tool, Analytical formula part, see figure below.

The zero in the formula section is for the horizontal plane creation. You calculate the matrix and you can see the plane in three dimensions.

Then to the geographical parameters, that have to fit the DEM extent without creating an excessively large geological plane.

We create the simulated geosurface, optionally view it in three dimensions and then have to export it in the Geo Analytical Surface (GAS) format, i.e. a jason format.

Plane rotation

We then pass to the Geosurface deformation tool, import the previously exported jason file and then apply a rotation to the plane around a N-S horizontal axis, by 35°.

Apply and then rotate by 45° around a vertical axis (plunge equal to 90°).

In this way we obtain a plane dipping 35° towards N135°.

Plane displacement to DEM extent

Now we locate the rotated plane to a geographical position that broadly fits with the DEM. I choose to use my qgSurf plugin for QGIS for quickly locating a point at the center of the used DEM, while knowing also the z value.

You see to the right the coordinates (x-y-z) of the point at the DEM center, showed within QGIS.

I copied and pasted these values in the simSurf displacement tab, so that the plane is displaced by the given delta-x, delta-y and delta-z amounts.

Done, after applying.


Save the geosurface as a VTK file and then you can see it in Paraview and use in the geoSurfDEM application. Note that the VTK file stores the plane as triangle mesh, without explicit attitude (i.e., dip direction and angle) information. So the local results calculated by the geoSurfDEM application are derived by the local geosurface triangle attitude stored in the VTK file. Using a simple plane obviously we expect the same results for all intersection points.

Input data preview

We see how are the DEM and the VTK plane data in Paraview.

You can import the DEM when in x-y-z format (could create with Saga), then applying a
Table to Point filter, while the VTK format is directly read from Paraview. Here a nadiral view.
Y axis represents the North.

And a lateral one, as seen from the South.

geoSurfDEM result

At the end, what are the results of the geoSurfDEM application?

We see them displayed in Paraview, by importing the resulting csv file and superposing on the DEM points and the plane surface. The results are symbolized by blue dots. You see them following the visual intersection between the plane with dip direction 135° and dip angle 35° and the DEM.

 

Always in Paraview we see, for a few records, that the corresponding point attitudes calculated by geoSurfDEM are as expected: 135°/35° for each point, since in this test case we were dealing with a geological plane.

----------

The code repository of geoSurfDEM is at https://gitlab.com/mauroalberti/geoSurfDEM

Executables for Linux are also availables for Linux (64 bit) at the code repository.

EDITS: 

2023-01-01: fixed broken llinks to images

Adding a Python module to QGIS Python in Windows with multiple Python installations

Both Python and (if I rememeber well) QGIS were originally created for Linux. Their use in Windows is somewhat complicated by the difficulties to adapt these tools to a different environment.
When working with an installation of QGIS in Windows created with the standalone installer, one may need to use Python modules that are not present in the default QGIS installation. Modules are generally installed in the folder ..\Python27\Lib\site-packages.
The installation can be confusing if you have also other Python installations, for instance one derived from the Python(x,y) distribution, from Anaconda or from ArcGis.
The QGIS Python installation from the standalone GGIS installer is not system-wide, and QGIS Python is separated and independent from the other Pythons.
After a bit of struggling, I found a way to install a Python module from the source files (tar.gz) in QGIS Python.

Source module

The Python module that I wanted to install is pyserial, that I downloaded as a tar.gz file from https://pypi.python.org/pypi/pyserial.
I uncompressed it in a temp folder, in my particular case:
C:\Temp\dist\pyserial-2.7\pyserial-2.7
This folder contains the required setup.py file, plus some other stuff.

Target QGIS installation 

The target QGIS installation is 2.10.1 Pisa 32bit (from standalone installer), with installation folder:
C:\Program Files (x86)\QGIS Pisa

In QGIS Python, the path to the standard Python module folder is:
..\QGIS Pisa\apps\Python27\Lib\site-packages.
Installing a module in this last directory, we have it recognized by QGIS Python.

Prerequisites

As a prerequisite, you have to be sure to launch from the DOS shell the QGIS Python and not other Python interpreters. To achieve that, one possible (but there are others) way is to add the path to the QGIS python.exe file as the first item in the system PATH.
In Windows 8, follow Control Panel -> System and Security -> System and then , by opening Advanced system settings, modify the Path variable (user or system are equivalent ways, I suppose, in my case I had the system one modified) by adding as the first voice:
C:\Program Files (x86)\QGIS Pisa\bin
i.e., the bin folder of the QGIS installation, where python.exe is contained.

Procedure

Being sure that from the DOS shell you are invoking the QGIS Python, you can then proceed as follows:

1) launch an administrator dos shell, to be sure to have write permissions in the target directory

2) in the dos shell, change directory to the uncompressed folder of the new module to install, (it should contain the setup.py file), in my case with the command:
  cd C:\Temp\dist\pyserial-2.7\pyserial-2.7

3) always in the dos shell, run the python setup.py install command on the module, in my case:
  python setup.py install --prefix="\Program Files (x86)\QGIS Pisa\apps\Python27"
Change path in the prefix option accordingly to your particular QGIS installation. Note two things:

• do not put the drive character at the beginning of the prefix path; 
• point to the Python27 folder, not to the  Python27\Lib\site-packages

Checking the results

In addition to the installation messages in the DOS shell, that should present no errors, you may then check that the module installation was successful by trying to import the specific module in the QGIS Python console, in my case:
import serial
You should receive no import error messages..

beePen: a tool for drawing freehand annotations in QGis

beePen is an experimental Python plugin for drawing freeand annotations and sketches in an ad-hoc layer in QGis. It is inspired by the corresponding tool in BeeGIS and re-use the code by Pavol Kapusta in his Frehand Editing plugin.

beePen allows to create an annotation layer, that is characterized by three fields, storing the user-defined width, color and transparency of the pen. These value are automatically inserted based on the user choice in the plugin window (Fig. 1).

Fig. 1

To draw annotations, first you have to open the beePen window with the "bee" command (see Fig. 2, second icon from the right), eventually create a new annotation layer from the beePen window (see Fig. 1) or use an old one, put in edit mode using the standard tool of QGis (command at the left in Fig. 2) and then draw after selecting the pen tool (rightmost command in Fig. 2).

Fig. 2

The conception of beePen is by Mauro Dedonatis (Urbino Univ., Italy), while the implementation is by Mauro Alberti.

Some minor enhancements in qProf

qProf is a QGIS plugin for the creation of topographic and geologic profiles. Some minor enhancements were added to qProf in the new, experimental releases 0.2.9 and 0.3.0.

In version 0.2.9 the following options were added:
- the calculation of the absolute slope along the profile; previously only the directional slope was calculated;
- the possibility to flip a profile horizontally (option "Reverse profile direction"), as well as to reverse the orientation of the x axis (option "Reverse x axes direction) were added.

See examples in Figs. 1-4 (below).

Fig. 1. Map of Mt. Alpi zone (Basilicata, Southern Italy), with profile in red (direction from left to right). DEM data: TINITALY.

Fig. 2. Topographic (top) and absolute slope (bottom) profiles (see map trace in Fig. 1).

Fig. 3. Same profile as in Fig. 2, but with profile direction reversed.

Fig. 4. Same profile as in previous figures, but with reversed x axes directions.

In version 0.3.0, released on 2015-03-08, the possibility of configuring the figure save parameters (width, resolution, font size, subplot configuration parameters) was added. These graphic parameters can be saved in a text file and subsequently loaded for being applied to future plots.
This option is available from Export - Figure. Fig. 5 represents the window for defining the parameters, saving or loading them, and also for saving the figure as a file in PDF, SVG or TIF formats.

Fig. 5. Window for saving a figure as a graphic file, and for configuring graphic parameters, and saving or loading them.

The two versions are experimental since the impact of reversing the profile on the geological plotting has not been fully tested. For geological plots, the suggested version is 0.2.8.

Gerrit Tombrink, PhD student at the Goettingen University (Germany), is thanked for proposing the implemented options.

Cross-section intersections of line and polygon layers with qProf in QGis

A new release of qProf, plugin for QGis, allows to determine the intersections of line and polygon layers, representing for instance faults and geological outcrops, on a vertical cross-section.
These new functionalities complement those already available for projecting geological attitudes and lines on a cross-section.
The resulting data can be saved as shapefiles and csv.

Example of cross-section representing geological polygon layer intersections with a topographic profile.

The plugin can be installed/updated via the plugin manager of QGis. Alternatively it can be downloaded from the QGis plugin repository:
http://plugins.qgis.org/plugins/qProf/
or from the Bitbucket repository:
https://bitbucket.org/mauroalberti/qprof/downloads

R is progressing fast in TIOBE Index

According the TIOBE Index for November 2014, R, an open source statistical language,  progressed from position 15 in October to position 12 in November, thus heading for entering in the "top 10". It is preceded by Visual Basic and Visual Basic .NET. For instance, Python is at position 7.
The much younger Julia language is at position 126 while MATLAB is # 24.
More details:
http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html

Changes in qgSurf plugin tools

Due to persisting plugin incompatibilities, related to Matplotlib, I've created a new qgSurf version, # 0.3.4. This version is made up by just two tools: "Best fit plane" and "DEM-plane intersection".
The removed 3D surfaces simulation and deformation tools are now available in a pure Python project, "simSurf", developed and tested in Linux Mint and published at: https://github.com/mauroalberti/simSurf.

Simulation and deformation of georeferenced geological surfaces: a pure Python implementation

A pure Python project for simulation and deformation of georeferenced geological surfaces is presented at:
https://github.com/mauroalberti/simSurf
It is an updated and Python-based, independent version of the simulation and deformation modules originally presented in the qgSurf plugin (vers. 0.3.3) for Quantum GIS. It is developed and tested in Linux Mint.
Two modules, "geosurface_simulation.py" and "geosurface_deformation.py" make possible to simulate and deform georeferenced surfaces (see example in Fig. below).

Example of a sheared and rotated sinusoidal surface with geographic parameters matching those of the Mt. Alpi - Mt. Raparo Aster DEM (Lucania, Southern Italy). The view is from NE to SW. Mt. Raparo is at the right. 3D visualization created with ArcScene (ESRI).

A MOOC on Numerical Methods with Python, starting August 18, 2014

A new MOOC, "Practical Numerical Methods with Python", with Lorena A. Barba and other instructors, has just started.
The link is: http://openedx.seas.gwu.edu/courses/GW/MAE6286/2014_fall/about

New version of qgSurf for QGIS

A new version of qgSurf, 0.3.3, has been released. It adds the possibility to use point or line layers as sources for the best-fit-plane calculation.
The plugin should work also in QGIS 2.2.0 for Windows 8 - 64 bit, even if the QGIS installer complains about a lacking dependency, Tkinter, a problem that I will investigate in the next days. If however you accept the installation, close QGIS and then reopen it, QGIS will present the plugin as installed and available. No problem of this kind has been observed in Windows Vista and Ubuntu.

Backward pathline calculation for steady vector fields in Quantum GIS

Calculating backward pathlines in steady vector fields is possible using the new version of VectorFieldCalc, a Python plugin for Quantum GIS. The pathline calculation is implemented via the Runge-Kutta-Fehlberg method (see references at the end).

For forward calculations, the user defines positive values of time steps and total times. For backward calculations, use negative values for both total time and time steps. An example for the last case, using the example data provided in the plugin folder, is below.

The result of the backward pathline calculation is presented in the figure below, with pathline points symbolised by increasingly negative total time, together with labels of the total times. The flows in this Antarctic glacier in Terra Nova Bay (data from D. Biscaro PhD thesis) are from top-left to bottom-right. Higher velocities of the flow field are in red, lower velocities in blue. Starting points are represented by red squares in the bottom-left section of the glacier.

 The plugin can be installed via the usual procedure for Quantum GIS, using the plugin manager.

For additional details (based on the previous version) you can see:
- Vector Field Processing
- 'Vector field processings': un plugin Quantum GIS per l'analisi di campi vettoriali 2D (in Italian)