Tuesday 11 November 2014

R is progressing fast in TIOBE Index

According the TIOBE Index for November 2014, R, an open source statistical language,  progressed from position 15 in October to position 12 in November, thus heading for entering in the "top 10". It is preceded by Visual Basic and Visual Basic .NET. For instance, Python is at position 7.
The much younger Julia language is at position 126 while MATLAB is # 24.
More details:
http://www.tiobe.com/index.php/content/paperinfo/tpci/index.html


Thursday 25 September 2014

Changes in qgSurf plugin tools

Due to persisting plugin incompatibilities, related to Matplotlib, I've created a new qgSurf version, # 0.3.4. This version is made up by just two tools: "Best fit plane" and "DEM-plane intersection".
The removed 3D surfaces simulation and deformation tools are now available in a pure Python project, "simSurf", developed and tested in Linux Mint and published at: https://github.com/mauroalberti/simSurf.

Friday 12 September 2014

Simulation and deformation of georeferenced geological surfaces: a pure Python implementation

A pure Python project for simulation and deformation of georeferenced geological surfaces is presented at:
https://github.com/mauroalberti/simSurf
It is an updated and Python-based, independent version of the simulation and deformation modules originally presented in the qgSurf plugin (vers. 0.3.3) for Quantum GIS. It is developed and tested in Linux Mint.
Two modules, "geosurface_simulation.py" and "geosurface_deformation.py" make possible to simulate and deform georeferenced surfaces (see example in Fig. below).


Example of a sheared and rotated sinusoidal surface with geographic parameters matching those of the Mt. Alpi - Mt. Raparo Aster DEM (Lucania, Southern Italy). The view is from NE to SW. Mt. Raparo is at the right. 3D visualization created with ArcScene (ESRI).




Thursday 28 August 2014

Tuesday 24 June 2014

New version of qgSurf for QGIS

A new version of qgSurf, 0.3.3, has been released. It adds the possibility to use point or line layers as sources for the best-fit-plane calculation.
The plugin should work also in QGIS 2.2.0 for Windows 8 - 64 bit, even if the QGIS installer complains about a lacking dependency, Tkinter, a problem that I will investigate in the next days. If however you accept the installation, close QGIS and then reopen it, QGIS will present the plugin as installed and available. No problem of this kind has been observed in Windows Vista and Ubuntu.

Friday 13 June 2014

Backward pathline calculation for steady vector fields in Quantum GIS

Calculating backward pathlines in steady vector fields is possible using the new version of VectorFieldCalc, a Python plugin for Quantum GIS. The pathline calculation is implemented via the Runge-Kutta-Fehlberg method (see references at the end).

For forward calculations, the user defines positive values of time steps and total times. For backward calculations, use negative values for both total time and time steps. An example for the last case, using the example data provided in the plugin folder, is below.





The result of the backward pathline calculation is presented in the figure below, with pathline points symbolised by increasingly negative total time, together with labels of the total times. The flows in this Antarctic glacier in Terra Nova Bay (data from D. Biscaro PhD thesis) are from top-left to bottom-right. Higher velocities of the flow field are in red, lower velocities in blue. Starting points are represented by red squares in the bottom-left section of the glacier.



 The plugin can be installed via the usual procedure for Quantum GIS, using the plugin manager.

For additional details (based on the previous version) you can see:
Vector Field Processing
'Vector field processings': un plugin Quantum GIS per l'analisi di campi vettoriali 2D (in Italian)


Wednesday 28 May 2014

Released first beta version of gvSIG CE 1.0.0 (64 bit OS)

From gvSIG CE developer announcement:

The first beta version of gvSIG CE 1.0.0 has just been released.
Downloads for Linux, Mac OS X and Windows here:
https://sourceforge.net/projects/gvsigce/files/beta-1/
All provided packages are for 64 bit operating systems.

Read the full announcement at:
http://gvsigce.sourceforge.net/wiki/index.php/GvSIG_CE_1.0.0_beta_1_release_notes

Sunday 25 May 2014

Plot of geological attitudes and traces in vertical sections along fold axes with Quantum GIS and qProf

The new released version of qProf (0.2.4), a Python plugin for Quantum GIS, allows to project geological attitudes and traces onto a vertical section along fold axes. On-the-fly reprojection of layers is supported.

The first step in using this plugin is creating a topographic profile by defining one (or more) source DEMs and a line layer that define the section, plus a line densify distance (generally of the order of the used DEM cell resolution) and an optional line order field (required only when the input geometry of the line layer is not sufficiently simple, see previous posts on the qProf plugin).
While for creating only topographic profiles the input line can be made up of many points and more than one DEM can be used, for geological projections the input line must be made up by just two points and only one DEM may be selected.
After having defined and created the profile using the tools in the "profile" tab, it is then possible to project geological attitudes or traces by using the commands in the "Project" tab.


Projection of geological attitudes

Geological attitudes will be stored in a point layer, with the measured dip direction and dip angle of the features in two distinct fields of the attribute table.
The commands are available from the "Geological attitudes" subwindow.
The projection of geological attitudes may be applied via different methodologies: (a) the nearest intersection point between the geological plane of a feature and the section plane; (b) using a common projection ("fold") axis for all features; (c) using individual axes for each attitude, with two numeric fields storing trend and plunge values.
If some data are selected, only those selected will be plotted, otherwise all layer features will be used. Labels can also be added to projected measures.
It is possible to cumulate in a single analysis session more sub-analyses, distinguishing them via different colors. Previous sub-analyses can be removed with the 'Reset geological attitudes' button.
Data can be saved as csv files or shapefiles.


A plot example, with attitudes labelled by original trend and plunge values, is in the figure below.



Projection of line traces

Line traces can represent the intersection of geological surfaces with the topography. When these surfaces are cylindrically folded, it is possible to projected the traces onto the section via a user-defined fold axis. The elevation information will be derived from the chosen DEM.
The commands are available from the "Geological traces" subwindow.
A line layer has to be chosen, with the name of the id field and a value for the line densification.
Trend and plunge of the projection axis will be defined, and a color chosen for the line plotting.
Different subanalyses, with different projection axes, can be differentiated by using different colors.
Previous analyses can be eliminated with the "Reset traces" command.
Results can be saved as text file, storing id-s-z values (where s is the horizontal distance from the section start) for each resulting point.



An example of the output is below. In this case, only geological traces are plotted. In general, both geological traces and attitudes can be plotted in the same section.



Installation

qProf can be installed from Quantum GIS via the plugin manager, or downloaded and unzipped in the Python plugin folder from:


For bug segnalation:
or write directly to alberti.m65@gmail.com (also for improvement requests or questions).
 

Friday 9 May 2014

Interactive map of small particle air pollution in world cities

In the Guardian web site an interactive map, based on the data of the World Health Organization, is available:
http://www.theguardian.com/news/datablog/interactive/2014/may/08/exposure-air-pollution-city-map
North America has lower pollution levels than in European states, with south-eastern European countries having the highest pollutions for Europe. India, China, Mongolia, Pakistan, Afghanistan, Iran present the highest levels in the world, together with many Middle East cities.

Monday 5 May 2014

Traduzione del manuale di QGis in italiano: vuoi contribuire con una donazione?

"QGIS è un software open source molto potente sviluppato interamente da volontari. Ma cosa sarebbe un software senza un'accurata documentazione? 
Una nuova versione di QGIS viene rilasciata ogni 4 mesi e di conseguenza la documentazione cambia ed è necessario aggiornare il manuale con le nuove funzionalità in modo che tutti siano in grado di sfruttarlo al massimo.

Siamo un gruppo di volontari che nel tempo libero crea, mantiene e traduce la documentazione di QGIS. Aiutaci con un piccolo contributo!"





Thursday 17 April 2014

Interesting compilation of GIS-related open source Python modules

A listing of many Python modules for GIS tasks, from spatial analysis to interpolation, projections, geodatabases and web mapping, is available at Python GIS Resources.

Monday 14 April 2014

Python 2.7 End of Life: 2020 or later

Life for Python 2.7 is extended to at least 2020, giving more time to developers to migrate their code to Python 3.
You can read more at: http://hg.python.org/peps/rev/76d43e52d978

Friday 28 March 2014

qgSurf now supports QGis on-the-fly projection

qgSurf is a plugin for QGis, that allows to calculate the best-fit-plane given a set of point with DEM-derived heights, or the intersection between a geological plane and a DEM. Recently it has added some basic tools for the creation of 3D analytical surface and their deformation, even if they are still at an experimental stage.
The new version, 0.3.1, adds support for on-the-fly projection, so that, for instance, it is easier to use satellite data providers as Bing or Google for geological calculations.


When using the Best-Fit-Plane tool, DEMs can be in lat-long, but in that case the project must be set to a projected, planar CRS. DEM elevations and projected planar distances must be in the same measure unit (e.g., meters).
On the other hand, the DEM-plane intersection tool does not work properly with DEM in lat-long, since it requires that the horizontal distance measure unit to be the same as the vertical one, impossible for DEMs with data in lat-long. Also mixing meters for horizontal distances with feets for elevations will produce erroneous results. It is safe however to have a DEM in UTM 32 (with x, y and z values in meters) and the project CRS in Lambert Conformal Conic.

The plugin can be installed from the QGis plugin manager, or downloaded from http://plugins.qgis.org/plugins/qgSurf/ or https://bitbucket.org/mauroalberti/qgsurf/downloads.
It works for QGis >= 2.0, and has been tested in Windows Vista and Ubuntu LTS 10.2.
Errors in Linux or Windows for the 3D geosurface simulation tools could possibly be related to an old installed Matplotlib version, and solved by updating this last.
 





Monday 10 March 2014

Please support the upcoming Vienna Code Sprint 2014


Please support the upcoming Vienna Code Sprint 2014

In the occasion of the upcoming Vienna Code Sprint 2014 [1], the GRASS GIS Project Steering Committee decided earlier this year to  officially join this code sprint, considering it as a great opportunity  for joint activities. More than 60+ developers from the most important OSGeo project communities will be joining the event.
While the GRASS developers are donating their valuable time, the community of enthusiast users (you!) may contribute with donations,  even symbolic, that will be used to cover out-of-pocket expenses of the  participants. Companies can also decide to sponsor specific tasks!  Please don't hesitate to contact us [2] (or Markus Neteler, neteler@osgeo.org) for further details.
As usual, all of the work done in the community sprint will be directly contributed back to the GRASS project for the benefit of everyone who  uses it. Our scope at the sprint is to publish a first release candidate  of the stable GRASS GIS 6.4.4 version as well as a tech preview release of GRASS GIS 7.

For your convenience, here our easy-to-use Paypal button:

For our alternative bank transfer option, please contact Martin Landa (landa.martin@gmail.com)

Thanks for your support!
The GRASS Developers Team




About GRASS GIS
The Geographic Resources Analysis Support System (http://grass.osgeo.org/), commonly referred to as GRASS GIS, is an Open Source Geographic Information System providing powerful raster, vector and geospatial processing capabilities in a single integrated software suite. GRASS GIS includes tools for spatial modeling, visualization of raster and vector data, management and analysis of geospatial data, and the processing of satellite and aerial imagery. It also provides the capability to produce sophisticated presentation graphics and hardcopy maps. GRASS GIS has been translated into about twenty languages and supports a huge array of data formats. It can be used either as a stand-alone application or as backend for other software packages such as QGIS and R geostatistics. It is distributed freely under the terms of the GNU General Public License (GPL). GRASS GIS is a founding member of the Open Source Geospatial Foundation (OSGeo).



Sunday 2 March 2014

qProf supports on-the-fly reprojection and multiples profile lines

The new version, 0.2.2, of this plugin for Quantum GIS has now support for on-the-fly reprojection of layers in a project. The profile will be created with the CRS set by the user in the Quantum GIS project.

Moreover, it also possible to use multiple lines for the profile creation, that will be merged into one for the profile creation. Since it could happen that the order of the single lines is not the correct one for the profile creation, it has been added the option of reading the correct order from an integer field defined in the attribute table of the layer (see figure below).

Example of layer with multiple lines (4), that are digitized with an incorrect order (see in the attribute table), but with the correct order set in the field 'id' of the attribute table (also used for the labelling of the lines in the map).

Profile created with the correct line order, set in the attribute table as in the previous figure.


A bug related to the saving of results as shapefiles in Linux has also been removed.

The new version can be installed by using the plugin manager of Quantum GIS, or by downloading it from the Quantum GIS  plugin repository at http://plugins.qgis.org/plugins/qProf/ or at https://bitbucket.org/mauroalberti/qprof/downloads.





Monday 6 January 2014

Creating and deforming analytical surfaces in Quantum GIS: experimental tools in qgSurf plugin

Imagine you want to create a georeferenced sinusoidal surface, using a trigonometric function, and then deform it via simple shear. This surface may represent a folded geological layer, lately sheared.
This operation is possible using the presented tool, implemented in qgSurf, a plugin for Quantum GIS. Its purpose is to allow these types of operations, precisely: a) the creation of analytical surfaces in a geographical space; and b) their deformation using a few, well-known deformation matrices.


Modelling of geological surfaces

 In GIS continuous surfaces are generally stored as lattices/grids (rasters) or as Triangulated Irregular Networks (TINs). Both present relative advantages and disadvantages, but more importantly they share two disadvantages: first, they are memory-less, in the sense of not storing any information on their physical or mathematical generation history, and second, they have finite spatial resolution, a limiting factor when dealing with function-derived surfaces, that have a theoretically unlimited spatial resolution. Consider for instance multiple scale folds in structural geology, with superposed folds of different orders, or, in geomorphology, ripples superposed on dunes. The finite resolution of grids and TINs does not allow to represent the spatial structure from coarse to fine scales.
"Great Sand Dunes National Park - the tallest dunes in North America. Photo © copyright by Jack Brauer.From: http://www.mountainphotography.com/photo/dunes-ripples/
On the other hand, point lattices and TINs have the advantage of being expressed by points. Transformations can by applied on points by using linear equations or equivalent matrices. Deformations can be expressed as matrices, that can be incrementally added to represent a set of successive deformations [1].

This plugin allows to save the generative and deformation history parameters of a geosurface by choosing its internal, experimental "Gas" format, a simple Jason format with the parameters written as Python dictionaries. From them, a lattice of points can be generated at convenience, with the user defined grid parameters, and the deformation matrices can be applied to the points, producing a new 3D surface via triangulations.


Module structure

In addition to the previous 'Plane geoprocessing' module, the plugin presents two new modules.
With the first module, named 'Geosurface simulation', analytical, georeferenced surfaces (here called geosurfaces) can be created using the same approach as in Saga Gis (Grid - Calculus - Function, as of Saga vers. 2.0.3), visualised and exported as VTK, Grass or the internal, experimental "Gas" (geological analytical surface) format, a Jason format used in this plugin for storing the geosurface parameters, instead of the discretised points as in the VTK and Grass formats.
With the second module, 'Geosurface deformation', geosurfaces saved in the Gas format can be loaded and transformed via displacement, rotation, scaling and simple shear.


Simulation of geosurfaces

2.5 D surfaces can be created as analytical functions of a, b coordinates: z =  f( a, b ).
Ranges for a and b values are defined, as well as the number of grid columns and rows, to be used for the generation of the 3D surface.
An analytical formula is provided, in order to generate the analytical surface.
Some examples could be:
  • sin( a * a + b * b )
  • a * b + 1000
  • cos( a ) * 200
Note that the chosen functions are used to create Numpy arrays, so word the functions according to the Numpy nomenclature in order to use existing Numpy functions.

The analytical surface is created after pushing the "Calculate matrix" button, and it can be viewed in 3D by using the "View as 3D surface" button (see example below).
Analytical surface with formula: sin(a*b) and ranges - 5 to 5 for both a and b, and grid columns and rows equal to 70.


After the analytical surface creation, this surface can be georeferenced by using the commands in the 'Geographic parameters' widget. The following parameters have to be defined:
  • the length and width of the georeferenced surface to be created, 
  • its rotation angle with respect to the x axis, 
  • the x and values of the lower-left corner surface ('x min' and 'y min').

The figure below illustrates these concepts.

A georeferenced analytical surface is created after pressing the 'Create simulated geosurface' button and again can be viewed as a 3d surface by using the 'View as 3D surface' button.
Sinusoidal function created with the qgSurf plugin. The visualization is based on Matplotlib.

From the 'Output' widget it is possible to save the geosurface in the VTK, Grass or 'Gas' format.
VTK and Grass formats are widely used formats that stores the parameters of the geometrical elements constituting a surface. In our case, they are the triangular faces defining the complete surfaces, by means of the coordinates of each points.
In the Gas file format, on the other hand, no geometrical information is stored as points or faces. The saved informations describe the analytical and geographical parameters as defined in the 'Analytical formula' and 'Geographic parameters', plus the deformational parameters when present (described in the following paragraph). From this information, a new geosurface can be created at will.


The deformation of analytical surfaces

Surface can be changed via displacements, rotations or strains, each one with its own matrix or vector representation [1]. Apart from displacement, that is calculated as a vector that is added to the initial point position, all the other types are expressed as matrices that are multiplied to the initial point positions in order to obtain the final one. Obviously more than a deformation type can be applied to the same original analytical surfaces, for instance a vertical simple shear followed by a displacement and then a rotation.

Analytical functions produce 2.5 D surfaces. Through subsequent deformations, such as rotation or shearing, they can become true 3D surfaces, where more than one z value is defined for a single x-y value pair (image below).
Sheared sinusoidal surface. Visualised with Paraview.
The currently implemented deformation types are:
  1. displacement
  2. rotation
  3. scaling
  4. simple shear (horizontal)
  5. simple shear (vertical)




Displacement

A geosurface can be moved in the space, without rotation or distorsion, by given offsets in the x, y and/or z directions.

The displacement is calculated as the sum of initial point and the shift vectors.

Rotation

A geosurface can be rotated around a rotation axis, characterized by given trend and plunge values, by a rotation angle ω.


The rotated position is calculated by multiplying the rotation matrix with the initial position vector (eqs. 3.11a-c in [2]).

where:
a11 = cos ω + cos2 α ( 1 - cos ω )
a12 = - cos γ sin ω + cos α cos β ( 1 - cos ω )
a13 = cos β sin ω + cos α cos γ ( 1 - cos ω )
a21 = cos γ sin ω + cos α cos β ( 1 - cos ω )
a22 = cos ω + cos2 β ( 1 - cos ω )
a23 = - cos α sin ω + cos β cos γ ( 1 - cos ω )
a31 = - cos β sin ω + cos α cos γ ( 1 - cos ω )
a32 = cos α sin ω + cos β cos γ ( 1 - cos ω )
a33 = cos ω + cos2 γ ( 1 - cos ω )

and α is the angle between the rotation axis and the x axis, β is the angle between the rotation axis and the y axis, γ is the angle between the rotation axis and the z axis, and ω is the rotation angle.
The angles between the frame axes and the rotation axis are automatically derived from the rotation axis trend and plunge values.

Scaling

The size of the geosurface is scaled along the frame axes by three scale factors, Sx, Sy and S(X, Y and Z in the figure below).

The transformation matrix is:

Simple shear (horizontal)

We consider a horizontal simple shear (parallel to the x-y plane) with angle ψ (psi), along a direction that makes an angle α (alpha) with the x axis, as in the figure below.

The parameters are entered in this window:

Following the matrix derivation in Ramsay and Huber (1983), p. 290, the transformation is given by:

where γ is equal to tan(ψ).
Note the minus sign in the term "-γ sin2 α": in Ramsay and Huber 1983, eq. C.14 the sign is given as positive, but it appears to be inconsistent with both the derivation and the practical application of the formula.


Simple shear (vertical)

A geosurface can be sheared in the vertical plane, by an angle ψ (psi), along a direction making an angle α (alpha) with the x axis.


The parameters are entered in this window:
The transformation is given by:
where γ is equal to tan(ψ).


Current limitations

The visualization of deformed geosurfaces may fails when using the plugin internal visualizer, based on Matplotlib. Apparently the problem is linked to some features of Matplotlib management of points, but at the moment the problem is not solved. Also in Grass the visualization of deformed 3D surfaces may be unsuccessful, even if the imported data is correctly visualised in 2D.
On the contrary, Paraview import and visualization appears to be correct and without problems, at least on the tested datasets.


Download and installation

This version is still experimental. It can be downoladed and installed autmatically from the QGis plugin manager, or downloaded from http://plugins.qgis.org/plugins/qgSurf/version/0.3.0/ and unzipped in the user Python plugin folder (e.g., C:\Users\mauro\.qgis2\python\plugins for Windows Vista).


References

[1] Ramsay, J. G., Huber, M. I., 1983. The techniques of modern structural geology. Volume 1: Strain Analysis. Academic Press, Inc. 307 pp.
[2] Allmendinger, R.W., Cardozo, N., Fisher, D. M., 2012. Structural geology algorithms. Vectors and tensors. Cambridge University Press. 289 pp.