GeoSolutions' Blog

Developer's Corner: Introducing Database Level Security in GeoServer

2012-01-30T07:51:00.000-08:00

During our work we support manu GeoServer Enterprise installations which pull data from a spatial database of some sort, normally via a connection pool, a tool that keeps database connections around so that we don't have to open and close them at every request (something that could be very expensive).

The pool accesses the database via a shared user, that all GeoServer requests end up using. Some requests only require data reading (WMS GetMap), others modify data (WFS Transaction), some even create new tables (RESTConfig data uploading for example).

The pool user must be able to perform all and any of the operations that GeoServer needs, meaning that more often than not it has very wide powers of what it can do on the database.

GeoServer built in security, as well as extensions such as GeoRepository, allow to control what specific users can do and shield the database from security issues.

However in some enviroment the preferred security management policy is to have security restrictions operate at the database level instead, with the pool user being given minimal rights (normally, to list and describe the tables, but without any actual access to them). This has some advantages:

the security is setup just once for the variety of applications that might access the database
each user can actually perform only the operations that he/she was allowed to, regardless of eventual bugs/security holes in the application level software
leverages the DBA expertise

GeoSolutions recently implemented the ability to use DBMS session startup and teardown scripts that can be used to alter the user accessing the database for the duration of the current request, turning back to the pool user when the request is complete.

These commands can be specified in the configuration User Interface while setting up the data store. For example, if we wanted to have each and every PostgreSQL session use the credentials of the current GeoServer user we'd use the following setup:

Different databases will of course use different commands, or custom, in house package calls, to setup the current session user. See the GeoServer documentation for more details on how this new functionality can be used.

We'd like to thank Astrium GEO-Information Services for sponsoring this improvement and sharing it with the GeoServer and GeoTools communities.

Application security is certainly one of the topics we like to deal with. There is of course a lot more to explore and improve, this topic is both rich and interesting. Want for example CAS or Shibboleth security in your GeoServer intallation? Maybe integration with Active Directory? Talk to us first!

The GeoSolutions team,

Improving GeoServer SQL Server support

2012-01-26T02:42:00.000-08:00

Dear All,

in recent times we were hired to improve GeoServer SQL Server support story.

The SQL Server store was created and maintained during spare time by Justin DeOliveira, however due to lack of production usage, and work time to pour on it, it failed to reach to the same level of robustness and speed as the best supported stores, such as Oracle and PostGIS.

Our work this week tried to close this gap with a number of little and big improvements that make the code run faster and in a more reliable way:

add support for connection validation (very important for SQL Azure, which is very keen on closing pooled connections in your face)
use binary encoding, instead of text, to transfer geometries from the database
support for data paging at the database level
make sure the rich database test suite we have in GeoTools is fully implented for SQL server, ensuring good support for use cases such as dynamic SQL views, proper date/time encoding in filters, and the like, both on the development series and on the stable series

Our develoment focused on testing the code against both SQL Server 2008 and SQL Azure. SQL Azure is the SQL database one can use in the Microsoft Azure cloud system: while it does look a lot like SQL Server 2008, it does not quite behave the same way in all cases, and requires a specific JDBC driver to work properly.

There are still some improvements missing on the table, such as geography columns support, but we're sure you'll be able to get more out of a production usage of GeoServer and SQL Server now.

Interested in sponsoring further improvements? Looking for professional support service that deliver for your group? Let us know!

The GeoSolutions team,

GWT-OpenLayers release 0.7

2012-01-23T07:53:00.000-08:00

Dear All,
we would like to inform you that the new release 0.7 for the GWT-OpenLayers project is available.

The releases is ready for the download and can be found here.

Notable improvements with this release are as follows:

Upgrade to GWT 2.4.0
Support to Google Maps V3
Improved source code formatting
Added and fixed several base methods and bindings such WMS Params, Layer methods

Alessio Fabiani has taken care of the 0.7 release as an active committer and administrator of the project.
We would like to thank all the other committers for their dedication and hard work!

Regards,
the GeoSolutions Team.

BRISEIDE Project, Geoserver, GeoNetwork and GeoBatch for the management of dynamic MetOc data

2012-01-17T03:10:00.000-08:00

Dear All,
In this post I wanted to talk about the work we are doing for the BRISEIDE european project.

The ambitious aim of the project, as stated on its website, is the delivery of:

time-aware extension of MetOc data models developed in the context of previous/ongoing EU INSPIRE related projects (e.g. in the context of GMES, eContentPlus)
application (e.g. Civil Protection) based on the integration of existing, user operational information
value added services for spatio-temporal data management, authoring, processing, analysis and interactive visualisation

Within the context of the project, GeoSolutions will work under the leadership of SinerGIS in order to provide near-real time ingestion, cataloging and publishing of meteorological data provided by the stakeholders to be used as inputs for running processes to perform, as an instance, fire propagation models in emergency situations.

The infrastructure we are setting up is depicted in the deployment diagram here below. Basic building blocks are as follow:

GeoServer for providing WMS, WCS and WFS services with support for the TIME dimension. It is worth to point out that it will also provide WPS capabilities.
GeoNetwork, for publishing metadata for all data with specific customizations for managing the TIME dimensions in the dataset (we are going to biriefly describe them later on)
GeoBatch, for performing preprocessing and ingestion in near real time of data and related metadata with minimal human intervention.

For this project we have customized the metadata indexing (thanks Lucene!) in GeoNetwork in order to be able to index meteorological model runs in terms of their run time as well as in term of their forecast times. Generally speaking the data we are dealing with is driven by a meterological model which produces daily a certain number of geophysical parameters with forecast valid for around 5 days, moreover some additional fire risk indexes are produced by processing these results in near real time. As days go by forecasts from different runs of the model are available as indicated in the picture below.

To achieve our gaols we have slightly customized the indexing configuration as well as the user interface in order to be able to make searches on run times and forecast times fast. If you are interested in having a look at one of the ISO metadata XML documents that we are publishing, here you can find an example here.

Here below you can find a diagram depicting the automatic ingestion flow we have created for the BRISEIDE project (actually, one of the few we have created) using the GeoBatch framework (we will soon release version 1.0 on which we are working in our own repository).

This flow makes extensive use of an orchestration Groovy script that implements specific business logic for the use case. Internally it also make use of various other atomic action for performing tasks like, publishing an ImageMosaic in GeoServer or its metadata in GeoNetwork.

Incoming files are composed by a compressed set of .asc (ascii images) files which are:

Converted into re-tiled GeoTiff images
Embedded overviews are added to each image

The groovy script produces an ImageMosaicCommand which is essentially an xml command which is sent to the ImageMosaicAction which:

Check for the layer existence on the target GeoServer
If success copy all the files to the target directory
Create the Store and the layer to contain the ImageMosaic
Configure the layer on the GeoServer using desired parameters
Produce an XML file with the ImageMosaic properties

The groovy script read the produced ImageMosaic output, enrich it with some other useful information then pass that object to the FreeMarkerAction which using a template and the passed data model will produce the xml metadata file (as described above).

The GroovyScriptAction ends sending this file to the next action which is the GeoNetworkAction which will send the metadata to the target GeoNetwork server using the desired (specified by configuration) options.

If you have questions about the work described in this post or if you want to know more about our services could help your organization to reach its goals, do not hesitate to contact us.

Regards,
the GeoSolutions Team.

GeoNetwork 2.6.4 con traduzione italiana disponibile!

2012-01-10T00:38:00.000-08:00

Salve a tutti,
dopo il ticket di rilascio di una versione di preview della localizzazione italiana per la versione di sviluppo 2.6.5 di GeoNetwork (si veda qui per maggiori informazioni), abbiamo effettuato il backport alla versione 2.6.4 stabile. Da questo indirizzo è possibile scaricarne il war. Questo war installerà GeoNetwork versione 2.6.4 con in più la localizzazione in italiano; la pagina principale porterà automaticamente alla versione italiana.

La migrazione di installazioni esistenti sarà effettuata correttamente solo se la versione di partenza è una di quelle attualmente supportate da GeoNetwork standard, ossia 2.4.3, 2.6.0 o 2.6.1. Per le nuove installazioni i dati in italiano saranno normalmente caricati durante il setup del database di GeoNetwork.

The GeoSolutions team,

WFS for the masses: adding support for paging and sorting in GeoServer

2012-01-03T01:50:00.000-08:00

Today we are going to introduce you our latest contribution to GeoServer, WFS paging and sorting for retrieving features.

First off, let's take a step back and see what sorting and paging support is available in the official OGC protocols:

Neither WMS 1.1 nor WMS 1.3 (or SLD/SE for that matter), have any ability to order the results so that features are painted in a certain order. If features can be organized in categories filters and FeatureTypeStyle elements can do the trick, but that won't work over continous fields
Same goes for WPS 1.0, which can return significant amounts of vector data that might be useful to page over
WFS 1.0 does not support either, WFS 1.1 supports sorting, WFS 2.0 supports sorting and paging via the sortBy and startIndex/maxFeature parameters

The last stable GeoServer release does not support WFS 2.0, and allows for sorting only on DBMS based stores. In our latest contribution to the stable series we removed all limitations concerning WFS paging and sorting support:

WFS 1.0 and 1.1 can now support sorting on top of each and every store kind, using the sortBy parameter as a vendor extension
WFS 1.0 and 1.1 support paging on top of each and every store, using the startIndex/maxFeatures parameters

Protocol and paging wise we back-ported the work added on trunk along with WFS 2.0 support, and then we merged in some previous work on generic sorting we did for the aggregating store.

Technically speaking, in case the store does not support sorting natively (e.g., shapefile) we gather the features into an optimized merge/sort algorithm that never keeps more than 1000 features in memory, and uses secondary storage to scale up to larger result sets.

But, enough talking, let's see some examples against the states demo layer, using the CSV as the output format for brevity.

First ten features:

http://localhost:8080/geoserver/topp/owsservice=WFS&version=1.0.0&request=GetFeature

&typeName=topp:states&outputFormat=csv&propertyName=STATE_NAME,PERSONS

&maxFeatures=10

FID,STATE_NAME,PERSONS

states.1,Illinois,11430602

states.2,District of Columbia,606900

states.3,Delaware,666168

states.4,West Virginia,1793477

states.5,Maryland,4781468

states.6,Colorado,3294394

states.7,Kentucky,4551524

states.8,Kansas,2477574

states.9,Virginia,6180651

states.10,Missouri,5117073

The next 10 features:

http://localhost:8080/geoserver/topp/ows?service=WFS&version=1.0.0&request=GetFeature

&typeName=topp:states&outputFormat=csv&propertyName=STATE_NAME,PERSONS

&maxFeatures=10&startIndex=10

FID,STATE_NAME,PERSONS

states.11,Arizona,3665228

states.12,Oklahoma,3145585

states.13,North Carolina,6628629

states.14,Tennessee,4829958

states.15,Texas,17122020

states.16,New Mexico,1379559

states.17,Alabama,4040587

states.18,Mississippi,2573216

states.19,Georgia,6457339

states.20,South Carolina,3486703

The first ten states with most people (sort on PERSONS, descending):

http://localhost:8080/geoserver/topp/ows?service=WFS&version=1.0.0&request=GetFeature

&typeName=topp:states&outputFormat=csv&propertyName=STATE_NAME,PERSONS

&maxFeatures=10&sortBy=PERSONS%20D

FID,STATE_NAME,PERSONS

states.47,California,29760021

states.39,New York,18235907

states.15,Texas,17122020

states.23,Florida,12937926

states.40,Pennsylvania,11881643

states.1,Illinois,11430602

states.48,Ohio,9980887

states.24,Michigan,9295297

states.43,New Jersey,7484736

states.13,North Carolina,6628629

The second page of the above result set:

http://localhost:8080/geoserver/topp/ows?service=WFS&version=1.0.0&request=GetFeature

&typeName=topp:states&outputFormat=csv&propertyName=STATE_NAME,PERSONS

&maxFeatures=10&startIndex=10&sortBy=PERSONS%20D

FID,STATE_NAME,PERSONS

states.19,Georgia,6457339

states.9,Virginia,6180651

states.37,Massachusetts,6016425

states.44,Indiana,5544159

states.10,Missouri,5117073

states.49,Washington,4866692

states.14,Tennessee,4829958

states.30,Wisconsin,4796441

states.5,Maryland,4781468

states.7,Kentucky,4551524

These modifications are already available in the GeoServer 2.1.3 release.

There is more work to be done in this area, ordering features before paiting them in WMS, and paging WPS results would both be nice additions. Interested? Let us know!

The GeoSolutions team,

Robust Clustering Solution for GeoServer

2011-12-29T11:26:00.000-08:00

Dear All,
in this post we'd like to introduce some work that we have performed in order to provide robust support support for clustered GeoServer deployments with an emphasis on publishing new layers in real-time.

As you might know there are various approaches with GeoServer that can be used to implement a clustered deployment, based on different mixes of data directory sharing plus configuration reload. However, these techniques have intrinsic limitations therefore we decided to create a specific GeoServer Clustering Extension in order to overcome them. It is worth to point out that what we are going to describe is designed to work with GeoServer 2.1 stable series.

In the picture below our approach is shown. We propose a robust Master/Slave approach which leverages on a Message Oriented Middleware (MOM) where:

The Masters (yes, we can have more than one, read on...) receive changes to the internal configuration, persiste them on their own data directory but also forward them to the Slaves via the MOM
The Slaves do not accept changes to their configuration from eithe REST or the User Interface, but are configured to inject configuration changes disseminated by the Master(s) via the MOM
The MOM is used to make the Master and the Slave exhange messages in a durable fashion
Each Slave has its own data directory which it is responsible for keeping it aligned with the Master's one. In case a Slave goes down when it goes up again he might receive a bunch of messages to align its configuration to the Master's one.
A Node can be both Master and Slave at the same time, this means that we don't have a single poinf of failure, the Master itself

Improved Clustering in Action

In the following we provide a few additional technical details on our solution, describing a deployment which we use for our tests (as such, it is designed to show every possible combination as opposed to be a best practice for deploy). We will refer to the following picture.

This deployment is composed by:

A pure Master GeoServer(s), this instance can only send events to the topic.It cannot act as a slave
A set of Geoserver which can work as both Master and Slave. These instances can send and receive messages to/from the topic. They can work as Masters (sending message to other subscribers) as well as Slaves (these instances are also subscribers of the topic).
A set of pure Slaves GeoServer instances whic can only receive messages from the topic.
A set of MOM brokers so that each GeoServerinstance is configured with a set of available brokers (failover). Each broker use the shared DB as persistence. Doing so if a broker fails for some reason, messages can still be written and read from the shared database.

We are now going to illustrate, step by step, how to publish a layer from a GeoTiff file using the GeoServer User Interface of the Pure Master instance. The resulting layer will be published on all the active GeoServers.

Manually create the GeoTiff store using the User Interface:

Publish the layer

Click save and check results on the clients

Now check the result using the LayerPreview:

As expected using the pure Master to publish the GeoTiff file, the resulting layer will be published automatically on the salve instances with no intervention. There is one thing to notice, we are not moving data around but only their configuration (styles included) since we are assuming that all instances sees the same resources with the same absolute paths, which is common in distributed and cluestered set ups where resources are shared among multiple servers, like for example network storage.

Conclusions
If you are responsible for administering a a series of GeoServer instances and/or you are publishing lots of data in real-time then this extension is a perfect fit for your organization.

In case you are interested in test-driving this extension in your own set-up, you might want to know that we are going to provide this extension free of charge to clients who will subscribe to our GeoServer Professional Services for the 2012 as well as to our partners. Contact us if you are interested!

Regards,

the GeoSolutions Team.

Merry Christmas and Happy 2012

2011-12-23T09:33:00.000-08:00

Regards,

the GeoSolutions Team.

Job offer @ GeoSolutions

2011-12-05T14:01:00.000-08:00

GeoSolutions is looking for talented software engineers to fill a couple of positions which would mainly involve designing and implementing web-based geospatial applications. Qualifications are as follows:

Working knowledge of web 2.0 UI frameworks like GWT, Ext-GWT, JQuery, Ext-JS
Working knowledge of Java (JEE and JSE)
Working knowledge of GeoServer and OpenLayers is a plus
Knowledge of web development with Python is a plus
At least 1 year of experience
Being fluent in English, both written and spoken

We offer a variety of contracts but, please, notice that our intention is to establish a long term relationship, therefore this is not a position for freelance consultants.

Working remotely is an option, although we will give priority to candidate closer to our office.

Please send a detailed resume together with a letter of presentation at jobs_at_geo-solutions.it.

The GeoSolutions team,

GeoSolutions porta il GeoServer a Valenzia

2011-12-04T16:35:00.000-08:00

Si è svolto con successo il workshop su GeoServer organizzato da GeoSolutions durante le settime giornate internazionali gvSIG a Valenzia. Sono stati molti i partecipanti alle giornate che hanno deciso di dedicare 2 ore alla scoperta di GeoServer.

Il workshop si è svolto in inglese durante una manifestazione spagnola dove la maggior parte degli interventi avveniva nella lingua nazionale, ma questo non ha fatto desistere i partecipanti che hanno mostrato al contrario un grande interesse.

Un ringraziamento alla gvSIG association per il sostegno al software open source manifestato attraverso l'organizzazione di un evento con lo scopo di condividere esperienze a livello internazionale e per il supporto dato a GeoSolutions nell'organizzazione del workshop.

The GeoSolutions team.

GeoServer and GeoSolutions @ gvSIG Days 2011

2011-11-30T01:32:00.000-08:00

GeoSolutions will attend the 5th edition of the gvSIG Conference, organized by the Regional Ministry of Infrastructure and Transport (CIT) from December 2nd until December 4th at the Feria Valencia Convention and Exhibition Center.

Ing. Simone Giannecchini will provide an introductive workshop on GeoServer, therefore make sure you take note of the exact time on the conference program and you make room in your schedule for attending the event.
Check this link for more details.

See you in Valencia!

The GeoSolutions team.

Serving Meteo data with GeoServer, GeoBatch and GeoNetwork: the LaMMA use case

2011-11-28T02:00:00.000-08:00

Dear All,
in this post I'd like to talk about the work we have done for the LaMMA consortium.

The Problem
The purpose of this project is to build a complete Spatial Data Infrastructure (SDI) to provide a spatio-temporal raster data processing, publishing, and interactive visualisation facility. This platform is candidate to substitute the current one which was already built leveraging on Open Source software but which was rather static and contained no OGC services.

The data that will be ingested into the system is generated by an existing processing infrastructure which produces a set of different MetOc models. Our goal is to manage the geophysical parameter (or variables) produced by the following models:

ARW ECM

3 Km resolution
9 Km resolution

50 Km resolution

The ingestion is started every day at noon and midnight, hence there are 2 run-times a day for each model at a certain resolution and the produced data contains different forecast times.
- ARW ECM (3 days with interval of 1h)
- GFS (8 days with interval of 6h)
The data is produced in GriB format (version 1).

Our Solution
Leveraging on the OpenSDI suite and specifically on the following components:

as well as some other well known Open Source project such as (Apache Tomcat, Apache Http server, Postgres) we provided an extensible and standard based platform to automatically ingest and publish data.

The infrastructure we have put together is depicted in the deployment diagram below.


Deploy diagram

This infrastructure has been designed from the beginning with the goal of being scalable in terms of supporting large number of external users since it is based on a GeoServer Master/Slave infrastructure where multiple slaves can be installed for higher throughput. Caching will be tackled in a successive phase.

As you can see we provided three access level for different type of users:

Admin can locally access to the entire infrastructure and add instances of GeoServer to the cluster to improve performances
Poweruser can remotely add files to ingestion and administer GeoBatch via Basic Autentication
User can look at ingested data accessing one of the GeoServer slave machines via Apache httpd proxy server. The load of these accesses is distributed between all available slaves.

As mentioned above, the main building blocks are as follows:

GeoServer for providing WMS, WCS and WFS services with support for the TIME and Elevation dimensions
GeoNetwork, for publishing metadata for all data with specific customizations for managing the TIME dimensions in the dataset
GeoBatch, to perform preprocessing and ingestion in near real time of data and related metadata with minimal human intervention

Using GeoBatch for ingestion and data preprocessing

In the LaMMA project the GeoBatch framework is used to preprocess and ingest the incoming GriB files as well as to handle data removal based on a sliding temporal window (currently set to 7 days) since it was a design decision to keep around for live serving on the last 7 days of forecasts.

Below you can find a diagram depicting one of the automatic ingestion flow we created for the LaMMA project using the GeoBatch framework.


GeoBatch ingestion flow example

The various building blocks comprising this flow are explained here below:

NetCDF2GeotiffAction reads the incoming GRIB file and produces a proper set of Geotiff perfoming on-the- fly tiling, pyramiding and unit conversions.Each GeoTiff represent a 2D slice out of one of the original 4D cubes contained in the source GriB file
ImageMosaicAction uses the GeoServer Manager library to create the ImageMosaic store and layer in the GeoServer Master. The created ImageMosaic contains proper configuration to parse Time and Elevation dimensions' values from the GeoTiff in order to create 4D layers in GeoServer.
XstreamAction takes an XML file and deserializes it to a Java object this is passed to the next action.
FreeMarkerAction produces a proper xml metadata file for publishing in GeoNetwork, using a pre-cooked template and the passed data model.
GeoNetworkAction published the metadata on the target GeoNetwork
ReloadAction forces a reload on all the GeoServer slaves in order to pick up the changes done by the master instance

This type of flow, (with a slight different set up) is used to convert and publish the 3 different incoming models.

The other type of flow is the remove flow which is a composed by the following building blocks:

ScriptingAction executes a remove.groovy script which will:

calculate the oldest time to retain
select older files to be removes
search and remove matching metadata from the GeoNetwork
remove collected layers and stores from the GeoServer Master catalog
delete permanently succesfully removed files

ReloadAction forces a reload on all the GeoServer Slave.

Using GeoNetwork for metadata management
We have customized the metadata indexing (thanks Lucene!) in GeoNetwork in order to be able to index meteorological model execution in terms of their run time as well as in term of their forecast times.
Generally speaking the data we are dealing with is driven by a meterological model which produces daily a certain number of geophysical parameters with temporal validity that spans for certain number of time instants (forecast times) in the future. In GeoNetwork we are currently creating a new metadata object for each geophysical parameter (e.g. Temperature) of a new model run; this metadata object contains multiple links to WMS requests for each forecast time, leveraging the TIME dimension in GeoServer (see picture below). Moreover the forecast times themselves are indexed so that advanced searches can be done on them.

If you have questions about the work described in this post or if you want to know more about our services could help your organization to reach its goals, do not hesitate to contact us.

The GeoSolutions team,

GeoServer e GeoSolutions al GFOSS Day 2011

2011-11-22T06:29:00.001-08:00

GeoSolutions sarà presente alla conferenza GFOSS DAY 2011 che si terrà a Foggia il 24 e 25 Novembre presso la Università degli Studi.

Oltre ad essere sponsor Silver della manifestazione, GeoSolutions sarà presente con un workshop gratuito introduttivo ed una presentazione su GeoServer che si terranno entrambi nella giornata del 25 (si veda per dettagli il programma della manifestazione).

Quindi, se volete essere aggiornati sugli sviluppi del GeoServer o se volete essere introdotti al suo utilizzo da uno dei suoi creatori non vi resta che farci visita a Foggia!

The GeoSolutions team.

Animate your maps with GeoServer Animator Tool

2011-11-10T07:15:00.000-08:00

In this blog post we'd like to introduce you to a new GeoServer feature, the WMS animator.

The animator follows in the footsteps of the WMS reflector, allowing a client to build an animated map with a relatively compact syntax.

An animation is built as a set of frames, and each frame is a separate WMS GetMap call, similar to the others in the set, but with a different value in one of the parameters.

The animator allows you to specify the parameter that will vary and its value via the aparam and avalues request parameter.

For example, let's say we have a dataset with time support enabled, in particular a small set of blumarble images for the different months in the year. A call to get a continuously animated progression would look like:

http://localhost:8080/geoserver/wms/animate?LAYERS=topp%3Aworld&aparam=time&avalues=2004-01-01T00:00:00.000Z,2004-02-01T00:00:00.000Z,2004-03-01T00:00:00.000Z,2004-04-01T00:00:00.000Z,2004-05-01T00:00:00.000Z,2004-06-01T00:00:00.000Z,2004-07-01T00:00:00.000Z&format=image/gif;subtype=animated&format_options=gif_loop_continuosly:true

The aparam in this request is the WMS time parameter, while the avalues list a number of possible times, in particular a few months of the year. Also notice the output format, asking for an animated GIF that loops continously. The result is as follows:

Time and elevation are obviously prime candidates to drive an animation, but in fact every WMS request parameter can be used to drive the animation.

For example, we could change the bounding box and have the map move:

http://localhost:8080/geoserver/wms/animate?LAYERS=topp%3Aworld&aparam=bbox&avalues=-10\,30\,30\,60,-9.9\,30\,30.1\,60,-9.8\,30\,30.2\,60,-9.7\,30\,30.3\,60,-9.8\,30\,30.2\,60,-9.9\,30\,30.1\,60&format=image/gif;subtype=animated&format_options=gif_loop_continuosly:true

The actual reason the animator was developed was to allow building animation of tuna catches by ocean area over different types of aggregation factors, that are passed down to a parametrized SQL view building all the little squares you see in the map.

Here is an example showing the evolution of catches of a particular tuna spieces over the years:

http://localhost:8080/geoserver/wms/animate?layers=UN_CONTINENT,GRID_G5,TUNA_YEARLY_CATCHES,FAO_MAJOR,UN_CONTINENT&

format=image/gif;subtype=animated&aparam=viewparams:YR_TA&avalues=1990,1991,1992,1993,1994,1995&

width=512&format_options=layout:message;gif_loop_continuosly:true&viewparams=CD_GEAR:801\,802\,803;FIC_ITEM:2496\,3296\,3305\,2498;QTR_TA:1\,2\,3\,4;OP:sum

As you can see some of the maps include the indication of the year in the lower left. This is a recent addition to the animator framework, it uses a text WMS decoration to embed in the image the current avalue. See the animator documentation for details on how this can be achieved.

At the time of writing the only possible output format is animated GIF, but other formats could be written and plugged in, for example AVI or Flash, and transition effects could be added too.

Interested? Let us know!

The GeoSolutions team,

GeoBatch Public Demo Available!

2011-11-10T00:09:00.000-08:00

Dear All,
in this post we'd like to show the first very simple demo of the GeoBatch framework. We have set up a demo instance on our demo server side by side with a GeoServer instance so that user can perform:

Upload via FTP, preprocessing and ingestion in GeoServer in real-time of GeoTiff files
Upload via FTP, preprocessing and ingestion in GeoServer in real-time of Shapefiles

Additional details on how to play with the demo can be found in the slides here below:

We are going to update this demo frequently with new features. Just to mention some of them, in next version we will also add automatic metadata indexing in GeoNetwork. So, if you are interested, please check the GeoBatch site regularly and subscribe to the GeoBatch users' mailing list here.

If you have questions about the work described in this post or if you want to know more about our services could help your organization to reach its goals, do not hesitate to contact us.

The GeoSolutions team,

GeoServer e GeoSolutions ad ASITA 2011

2011-11-07T02:30:00.000-08:00

GeoSolutions sarà presente alla 15esima conferenza nazionale ASITA 2011 che si terrà alla Reggia di Colorno, Parma nei giorni dal 15 al 18 Novembre 2011.
In particolare il giorno 18 Novembre l'Ing. Simone Giannecchini, uno dei fondatori di GeoSolutions e membro permanente del comitato di controllo di vari progetti Open Source, discuterà delle potenzialità del prodotto GeoServer durante la Sessione 31, "Cartografia: gestione, aggiornamento, visualizzazione".

Ci vediamo ad ASITA!

The GeoSolutions team.

GeoSolutions è sponsor della conferenza GFOSS DAY 2011

2011-11-04T01:06:00.000-07:00

GeoSolutions è orgogliosa di annunciare il suo supporto come silver sponsor al GFOSS DAY 2011 che si terrà a Foggia nei giorni di 24 e 25 Novembre 2011.

The GeoSolutions team.

ImageI/O-Ext 1.1.2 Released

2011-11-03T01:26:00.000-07:00

Dear all,
GeoSolutions is pleased to announce the ImageI/O-Ext 1.1.2 release. With respect to 1.1.1, it adds a few bug fixes as well as improvements for the Tiff reader as well as on how we manage file memory mapping to provide a nice performance boost on remotely mounted file systems, e.g. NFS.
Release artifacts have been deployed on the GeoSolutions maven repository, as well as on the OSGEO one.

It is worth to notice that we are right now discussing with the GeoServer community for switching the GeoServer 2.1.x branch to this new ImageI/O-Ext release (the unstable trunk version already leverages on this newest ImageI/O-Ext) .

If you have questions about ImageI/O-Ext or if you want to know more about our services could help your organization to reach its goals, do not hesitate to contact us.

Regards,
the GeoSolutions Team

Time and elevation support in GeoServer

2011-10-27T08:30:00.000-07:00

Busy times in GeoSolutions and in GeoServer land! We just finished adding support for WMS time and elevation in GeoServer WMS on trunk, with plans to backport it to the stable 2.1.x series after the 2.1.1 release.

Configuring and using dimensions is performed, as tradition, by means an easy to use GUI. Just take a layer that has time information (or elevation), like this "storm tracks" layer:

Then switch to the new Dimensions tab and configure time support as follows:

Submit and voilà, you have your time enabled layer. Looking at the capabilities you can see GeoServer is automatically extracting the list of values for you (but you could also ask it to return just an interval, for brevity):

If you now ask GeoServer for the WMS layer you'll get only one track, the last one, also known as the current data:

But then again, you can ask for a month worth of data adding &TIME=2004-08-03/2004-09-03 to your request:

If you really want you also cherry pick the times you want to see, for example with &TIME=2000-08-03T16:00:00.000Z,2000-08-13T16:00:00.000Z,2000-08-17T10:00:00.000Z,2000-08-19T16:00:00.000Z:

For now we're added time and elevation support, but we're getting started with raster data and we should be landing that support soon too.

Interested in multidimensional data management advances? Let us know!

The GeoSolutions team,

GeoServer News: Hiding layers from the capabilities document

2011-10-12T04:21:00.000-07:00

Hi all,

in this blog post we would like to talk about a new functionality that we has just introduced in GeoServer, the layer "advertised" flag.

As you know, ignoring security settings for the moment, all enabled layers configured in GeoServer are made available though all the OGC services for which they make sense: vector layers are published automatically to WMS and WFS, raster layers to WMS and WCS.

This is a fine default behavior, but in some cases one would like that the layer was there, but without making the layer known to the general public. Tipical use cases:

you want to expose a set of layers as a layer group, but you don't want to publish the layers separately
some application needs to setup a temporary layer, maybe the result of some computation, without having the whole world know about it
the administrator wants a layer to be available for testing, but not have the whole world hit it yet (maybe because the layer configuration/styling is not done)

Up until GeoServer 2.1.1 the above was only partially possible by making the layer available only to certain users via the security setup, but in 2.1.2 it got just a lot easier, there is a new publishing flag, "advertised", that you can uncheck to hide the layer:

Once un-ckecked the layer won't appear anymore into the capabilities documents, nor in the layer preview table, but will still be available for all other service requests.

There are other things that could be done to improve layer management, such as attribute renaming, easier association with styles, publishing of a layer only towards certain OGC services and the like. Interested in getting any of that happen today? Let us know!

Regards,
the GeoSolutions Team.

GeoServer Aggregating DataStore: Aggregate and republish from several data sources

2011-09-26T03:26:00.000-07:00

When it comes to publishing data with GeoServer the most common case is to publish layers that one owns and that can store and serve fully from the local premises.

There are however cases in which a dataset with a uniform structure is handled by several different entities, each one making it available to others, but with no unified view of the data itself.

This might be due to geographical distribution of the stakeholders, separate ownerships, administrative subdivisions: a typical case is a set of local public administrations, all publishing the same, or similar, data set in a independent way.

For this use case we have developed the aggregating data store, a GeoTools store in which features pertaining to the same layer/feature type are coming from several different other stores: the store unifies all of them in a centralized view that can be accessed as a seamless whole.

The store requires the source types to be very similar, but not necessarily identical:

they can have a different layer names
may lack some attributes, or have some that are not shared with others
may not have a perfect type match (e.g., the same attribute can be an integer in one store and a string in another, a conversion to unify the two will be attempted).

A "default" store is appointed that defines the structure of the whole, and every other feature is remapped onto that structure.

Each data access is performed in parallel in order to minimize data loading time, and the store can be configured to tolerate the occasional disappearance of one of the data sources.

This new store is now made available as a GeoServer community plugin for the trunk series, and can be downloaded from the nightly builds repository along with a GeoServer trunk build that allows to use it.

We are still not managing attribute renaming, though it would be quite a nice feature to add. If you are interested in the topic contact us and let's see how we can work together to improve the store.

Regards,
the GeoSolutions Team.

GWT-OpenLayers release 0.6

2011-09-19T13:59:00.000-07:00

A lot of improvements and some initial alignment with OpenLayers 2.11 have been made on the fresh release of GWT-OpenLayers library.

Actually the developers are also evaluating a brand new refactored showcase and a full alignment with OpenLayers 2.11.

The releases are available for the download here:

https://bitbucket.org/gwtopenlayers/gwt-openlayers/downloads

On release 0.6 the major improvements are:

Implementation of OL client Icon
The possibility of removing layers from the map and changing the z-level
New zoom and pan controls for the map
Implementation of OL Dot Per Inch
Implementation of OL Styling
Implementation of OL Graticule
Improvements to OL Controls
Some nice Vectorial features stuff, along with measurements and geometric operations
Navigation History
Snapping
GML improvements
WMS Operations improvements
Markers
Google, VirtualEarth, Bing and OSM Layers improvements
TileManagerService Layers
PopUps improvements
Protocols improvements
A new showcase and a lot of examples

Alessio Fabiani has taken care of the 0.6 release as an active committer and administrator of the project.

Regards,
the GeoSolutions Team.

GeoSolutions Talks @ FOSS4G2011: the slides

2011-09-14T08:35:00.001-07:00

Dear All, here we go with the slides of our presentations at FOSS4G, for those that could not attend, and for everybody else that wants to get all the details of the presentations as soon as possible on their computer.
GeoServer

GeoTools

Raster data in GeoServer and GeoTools: Achievements, issues and future developments

The slides pack an incredible amount of ready to use information for all GeoServer and GeoTools users, do not miss them :-)

If you liked the styling SLD presentation we also setup a GeoServer publishing the example data set:

SLD styling of vector and raster data
Geometry transformations
The data directory, ready to use, with data and styles (mind, geometry transformations will work only in GeoServer 2.2.x)

Any topic that you want to explore in more details? The presentations wet your appetite and you crave for more content? Let us know!

GeoSolutions Talks @ FOSS4G 2011

2011-09-12T03:00:00.000-07:00

Dear All,
it looks like we will have some busy times at FOSS4G this week.
If you check this search you can see a few interesting talks that we will give, let's group them by topic:

GeoServer

The State of GeoServer Wed, 09/14/2011 - 1:00pm - 1:30pm
GeoServer on steroids Wed, 09/14/2011 - 1:30pm - 2:00pm
Advanced cartographic map rendering in GeoServer Wed, 09/14/2011 - 2:00pm - 2:30pm
Raster data in GeoServer and GeoTools: Achievements, issues and future developments Wed, 09/14/2011 - 3:00pm - 3:30pm
The status of GeoServer WPS Thu, 09/15/2011 - 2:00pm - 2:30pm

GeoTools

Raster data in GeoServer and GeoTools: Achievements, issues and future developments Wed, 09/14/2011 - 3:00pm - 3:30pm
Status of the GeoTools project Thu, 09/15/2011 - 10:30am - 11:00am

It is worth to point out that Wednesday there will be a full immersion on GeoServer with 3 talks in a row that will go from a general overview of the project to an in-depth analysis of rendering and production set up.

Are you curious to know more about the latest developments we have done? Interested in knowing how we can help your organization reach your goals? Contact us!

The GeoSolutions team,

Developer's Corner: Audit your GeoServer installation with the new monitoring extensions

2011-08-16T06:37:00.000-07:00

When running one or more GeoServer in production it is often important to keep tabs on the server and its workload.

The monitoring community module, available among the nightly builds, allows to track requests flowing into the server and expose basic statistics about the traffic in terms of OGC services being used.
Configured in "history" mode it also stores all relevant request details into a database, allowing other tools to perform detailed usage analysis in post processing.

This is great, but has some limitations. In particular, but storing all request details into a single database table the speed at which the new request details are inserted decreases over time in most relational databases, adding overhead to GeoServer as well, especially during peak loads.

We recently developed an alternative storage that uses plain text files instead, with file rolling when the file reaches a configurable record count as well as every time midnight is reached.

All the request details are first collected and fed into a queue that is then read by a separate thread doing the actual logging: this ensures there is no significant overhead and that the logging happens without slowing down the main thread.

Separate applications, running offline periodically, can then grab the files generated and compute whatever statistics are deemed necessary without impacting on GeoSeGeoServerrver's activity.

The logging by default builds XML files with all the details of a request, including the body of posts requests, as shown in this example file. However the Freemarker templates driving the logging allow the administrator to perform a completely different kind of logging, maybe duplicating the Apache log format or generate JSON documents instead.

See the online documentation for details on how it's possible to configure the logging module.

Interested in hearing more about GeoServer performance monitoring and auditing? Let us know!

Regards,
the GeoSolutions Team.

Developer corner: Compact choropleth map styles with the Categorize function

2011-08-03T03:03:00.000-07:00

One common task in cartography making is setting up a choropleth map, that is, a map showing different shades of colors or patters for different categories of a statistical attribute, such as population.

GeoServer example data directory ships with two such examples using an old USA states map having, among others, population count per state. The two following maps should be familar, the first is using the population style, the other the pophatch style:

The original styles use a typical SLD 1.0 setup to build choropleth maps, that is, use rule filters to tell apart the categories.
This is fine, but also quite verbose, especially since the only thing that changes between one rule and then next is a color or a graphic size.

SE 1.1 introduces the Categorize function, mainly as a way to categorize raster data, but in such a way that it can be used against vector data as well.
GeoServer and GeoTools offer the possibility to use the categorize function in SLD 1.0 as well, calling it like any other filter function.

This is the polygon symbolizer using the categorize function to select different colors based on the number of people in the state:

The function call should be read as "categorize based on the PERSONS attribute, use color red if we have less than 20M people, green between 20M and 40M, and blue above 40M".

As you can see the expression is quite more compact. Indeed the color based SLD goes down from 96 lines to 56 lines, whilst the hatch based SLD goes down from 124 lines to 69 lines.

What about performance, you might wonder? If you have few rules, say, less than 10, it should be more or less the same, whilst if you have a very large style, with hundreds of these rules, you should expect to see an improvement, the more visible the more rules you have.

In the next installment I'll show you how to have a compact style in case you want continuous changes of color based on the attribute value using the interpolate function instead.

Do you also have an interest in filter functions and their abilities? Or do you have a style that is growing beyond control and with a less than desirable performance? Let us know!

Regards,
the GeoSolutions Team.

Oceanographic Database, managing oceanographic data with GeoServer and GeoBatch

2011-07-25T02:38:00.000-07:00

Dear All,
in this post we'd like to introduce some work we have done recently at GeoSolutions towards an applications that is able to ingest in real-time, manage, visualize, retrieve and chart data acquired by oceanographic and meteorological sensors (like sensors mounted onboard underwater gliders as an instance) over time. We called this application, Oceanographic Database.

The infrastructure we have put together is shown in the picture below.

Summarising, we can identify the various components and their roles as follows:

Data is ingested via GeoBatch in near real-time with proper prepocessing
GeoServer is used in conjunction with some custom REST extensions for performinng visualization, dissemination and export to Google Earth
A REST framework wrapping on an open source charting library is used to generate charts on the fly from the ingested data.

In the following videos we are going to describe the main functionalities of the infrastructure.

The first video below shows the basics of the front-ed as well as the following functionalities:

basic and advanced filtering for visualization
creation of charts from ingested data
search and download of data
export in Google Earth with support for temporal animation and underwater visualization

The second video below covers the extensive charting functionalities.The O&M Database is capable of plotting 2D charts of the acquired geophysical parameters like:

Temporal evolutions of a certain parameter
Evolution of of a certain parameter over a depth range
Correlation of two parameters in a certain spatiotemporal range
Scatter diagram of a parameter with JET colormap

Going into details, the O&M Database allows users to filter data and then move onto the Diagrams Panel for selecting the parameters to plot as well as the type of diagram, Notice that charts are plotted in real time using the live data as the are stored in the sytem. This makes the tool a perfect mean to monitor the acquisition of data since very refined query can be used to plot chats of specific subsets of the data.

The third video below shows the O&M Database Data Download capability, which allows users to search for the original data and then to download them in the original format. As an instance we can search for Glider's data acquisitions and download them as netCDF files.

The forth video below shows the experimental Temporal Animation Capability of the O&M Database that allows users to perform simple temporal animations to visualize the temporal evolution of the acquisitions on the map.

The application is entirely built exploiting Open Source frameworks and libraries. Our intent is to make it availabe to our customers as part of our Enterprise Support Plans as Open Source under the GPL license (the proper license is still under evaluation).

Are you curious to know more about the features of the Oceanographic Database? Interested in knowing how we can help your organization reach your goal? Contact us!

The GeoSolutions team,

Preview: GeoRepository, advanced authorization manager for GeoServer

2011-07-25T01:10:00.000-07:00

Dear All,
in this post we would like to introduce an application which we have been working lately here at GeoSolutions in order to cope with the requirements coming from enterprise deployments in terms of Authentication and Authorization for GeoServer, which we have called GeoRepository since we want to make it grow even further in the near future.

Georepository short introduction

View more presentations from GeoSolutions

Whoever has been facing the technical problems involved with deploying GeoServer inside an enterprise infrastructure following the SOA knows in such context Authorization and Authentication capabilities are usually provided by external services in order to centralize A&A services, fostering reuse of existing capabilities, minimize risk of duplication of functionalities and as well as to avoid duplication of users' credentials. Moreover in such context Single Sign On (SSO) services are usually a must-have.

GeoServer comes with an integrated security subsystem that is able to handle both Authentication and Authorization which is quite powerful and customizable, however we needed even to extend it further in order to be able to provide fine grained control over users' authorizations as well as in order to better integrate with existing A&A services.

A typical and yet simple usage scenario for GeoRepository is depicted here below.

In these setups GeoServer is normally not exposed directly, there is normally a load balancer as well as an Apache configured to perform duties such as single sign on authentication, SSL encryption and connection with external monitoring tools such as Nagios or Hyperic.
As suggested the authentication is performed by Apache HTTPD via LDAP; GeoServer will therefore be responsible only for the authorization based on the current user. In order to do this, GeoServer communicates with GeoRepository in order to retrieve users' authorizations query limits (we will get to this later) and then applies them accordingly to the incoming requests (thus avoiding post-encoding filtering, see also this white paper comparing internal vs external security systems).

In order to make this approach possible GeoServer was modified by Andrea Aime, the author of the existing Security Subsytem as described in the picture below. GeoRepository obtains users' credentials from LDAP and allows administrator to decorate them with additional parameters which are specific for restricting GeoServer access to services and data (we will describe them shortly).

Now that we have seen the 10000 feet view let's delve into details of how us know provide additional details on how GeoRepository seamlessly applies specific geospatial access policies to one or more GeoServer instances

The possible types of authorizations that can be applied using GeoRepository are controlled via generic rules with priorities , are shown below.

Each rule comprises of:

User
Profile
Instance (since GeoRepository can control more than one instance of GeoServer, the typical enterprise scenario having a cluster of machines serving geospatial requests)
Service and Request which can be used to control grants on OGC services
Workspace which can be used to control grants on GeoServer workspaces
Layer which can be used to control grants on GeoServer layers
Grant
- ALLOW grants access if the rule is satisfied, DENY denies access if the rule is satisfied
- LIMIT applies only when we have spatial restrictions available, in this case if the ruse is satisfied we basically perform an allow with spatial restrictions.

For ALLOW and LIMIT grants we can, under some conditions, set also additional limits which are called details.

Available detail levels are:

Layer Details allows us to specify refined rules for a specific GeoServer layer. As an instance it is possible to override the default style or to filter the available styles' list. It is also possible to specify CQL filters for reading and/or writing (when possible) GeoServer layers' attributes.
Layer Attributes Details allows us to limit access to vector layer attributes. By default all attributes are readable/writable, but with this capability we can hide some of them or make them read only.

It is worth to point out that GeoRepository interprets rule by priority: higher priority rules (represented by a smaller integer number) are applied before rules at lower priority. Like in IPTABLES, the first match is considered valid and then executed.

The possibilities introduced by having GeoRepository work right next to GeoServer are quite a lot, let us list some of them:

Fine grain control over which services, layers, workspaces, operations (of a service) a user can access
Fine grain control over which geospatial area for a layer (being it raster or vector) a user can access
Fine grain control over the WMS style that a use can apply for a layer with the possibility to perform style substitution on a per request basis depending on the credentials received
Fine grain control over which attribute (for vector layers) or bands (for raster layers) a user can access (this way it is possible to hide/show sensible attributes depending on credentials)

As an example of full versus limited access to resources, check the difference in the following two layers previews. First one show administrator's preview, second one shows unprivileged user's preview, where some the attributes are hidden, the default style is forced by the access control and the geometries outside the restriction area have been .

We should set up an inline demo soon, meanwhile, if you want to know more about the features we currently support or about our roadmap, please, do not hesitate to contact us.

Regards,
the GeoSolutions Team

ImageI/O-Ext 1.1.1 Released

2011-07-21T03:35:00.000-07:00

Dear all,
GeoSolutions is pleased to announce the ImageI/O-Ext 1.1.1 release. With respect to 1.1.0, it adds support for the IDRISI Raster Format. This format should be soon be available in GeoTools and GeoServer trunk.
Release artifacts have been deployed on the GeoSolutions maven repository, as well as on the OSGEO one.

Regards,
the GeoSolutions Team

Developer corner: taming a massive style and get a performance speedup in the process

2011-07-13T05:27:00.000-07:00

Hi all,
recently a customer contacted us with an interesting problem: how do I display a large dataset fast, with the requirement that we want every feature with a different color (unique color approach)?

I cannot show you the original dataset, but let's consider Natural Earth level 1 administrative subdivision: all provinces in the world. And let's say we want to give each of them a unique color, considering the are 3791 of them.

The common SLD approach would make for a Rule with filter, and symbolizer, for every single one of them. Assuming 15 lines of SLD per rule, that would make for a SLD with roughly 57000 lines of XML. Ugh! So large that nothing will allow the upload of it... double ugh!
As icing on the cake, for each feature on average 1900 rules would have to be evaluated in order to find the right color. Are you ready to way minutes for you map to be drawn?

Enter SE 1.1 and the recode function. Recode allows a compact representation of a style in which one attribute value is to be associated to a particular color, or width, in a very compact form.
Here is a portion of the SLD for the example at hand:

The full SLD can be downloaded here. It's around 7600 lines, still quite big, but remember it condenses in a compact form over 3700 rules.

Now, the function surely makes the SLD smaller, but was still quite slow. So we spent some time optimizing it for the common case where all the values and colors are static values, getting a tremendous speedup.

How fast is it now? Brace yourself: by the naked eye, it's as fast as the plain style that assigns the same color to all the polygons, the style that has just a single rule.

A larger version of the same map is available here. This large version took half a second to generate.

We hope that similar optimizations can be applied, in the future, to other SE useful functions, such as Categorize and Interpolate. Interested in the topic? Let us know!

Regards,
the GeoSolutions Team.

Developer's Corner: GeoNetwork-Manager, super-simple Java client library for GeoNetwork

2011-07-12T08:35:00.000-07:00

Dear All,
we just wanted to spend a few words about a Java library we have just released under the MIT license for talking to the GeoNetwork Open Source Project. We have named this library GeoNetwork-Manager (once again, a lot of brainstorming behind that name, really...).

Basically the goal of this library is to provide a simple yet usable way to interface a Java application with the GeoNetwork that would actually ease the work of the developers hiding away from them the relative complexity of theCSW protocol without introducing a complex communication layer.

Here you have some basic facts about the library:

minimum set of external dependencies, namely apache-http-common and jdom
dead-easy API, see examples here
works with GeoNetwork 2.1 and 2.0.X
allows to performa basic operations:

Insert
Reset Privileges
Delete

GeoNetwork-Manager is open source and licenced under the MIT License. This library draws its essence from the efforts endured inside GeoSolutions in various projects. Partial funding has been provided by the BRISEIDE project..

If you have questions or if you just want to talk to us about the using the library in your project, please, subscribe to the mailing list here. In any case, do not hesitate to contact us.

Regards,
the GeoSolutions Team.

GeoSolutions @ FOSS4G 2011

2011-07-04T11:58:00.000-07:00

Dear All,
it looks like we will have some busy times at FOSS4G this year.
If you check the schedule here you can see a few interesting talks that we will give, let's group them by topic:

GeoServer

GeoTools

JAITools

~~jai-tools: advances in Java image processing~~

Well, it is encouraging to see that people are interested in what we are doing, all the effort we put into doing Open Source software is producing value for the larger community.

Are you curious to know more about the latest developments we have done? Interested in knowing how we can help your organization reach your goals? Contact us!

The GeoSolutions team,

GeoServer 2.1 workshop available for free!

2011-06-26T23:56:00.000-07:00

Dear All,
a quick post to just let you know that GeoSolutions is sharing an introductory workshop on GeoServer 2.1.

The material comprises of:

a lightweight Ubuntu-based ISO
a preconfigured GeoServer data directory
the workshop itself, as a set of html files

The ISO file can be run either using Virtual Box or the VMWare Player. As an alternative you can burn it on a CD and you can use it as a Live Ubuntu or install it on your system.

Once you get it up and running, the installation contains:

An instance of the GeoServer 2.1.x stable nightly .
PostgreSQL with Postgis
UDIG

The topics covered in this workshop are as follows:

Installing and running GeoServer: describes how to install and run GeoServer

Adding Data to GeoServer : explains how to manage the GeoServer Data Directory as well as how to add the base data types into GeoServer, like ShapeFile, GeoTIFF as well as PostGIS data. Examples and informations about SQL Parametric Views and WMS cascading support are also provided.

Pretty maps with GeoServer: describes how to manage the GeoServer maps visualizzation. All aspects related to styles, layer groups and other interesting GeoServer features affecting the WMS protocol will be introduced

GeoWebCache Integration: provides additional information about the tighter integration between GeoWebCache and GeoServer .

Google Earth and Maps support: introduces GeoServer support for KML and Google Earth. The reader will learn how to view GeoServer data in Google Earth, and will be exposed to some of the more advanced features of KML output.

You can download the workshop from here:

In case you find errors or issues, or in case you want to know more about our services with regards to training, please contact us!

The GeoSolutions team,

ImageI/O-Ext 1.1.0 Released

2011-06-21T02:53:00.000-07:00

Dear all,
GeoSolutions is pleased to announce the ImageI/O-Ext 1.1.0 release. It replaces the 1.1-RC1 release (by also adding BSB and CADRG RPFTOC plugins through GDAL).
Changes with respect to 1.0.x series can be summarised as follows:

Out-of-the-box Support for GDAL 1.7.3, which means no more patches are needed for GDAL Java bindings in order to access it from ImageI/O-Ext.
BigTiff support, breaking the 4GB TIFF limit.
EnhancedImageReadParam support. It extends the standard ImageReadParam by implementing Cloneable (used when supporting MultiThreading read operation) and a new DestinationRegion param to support oversampling/subsampling without specifying sourceSubSampling parameters. This may be used when dealing with readers which internally take care of performing subSampling/overSampling, such as the GDALImageReader
Experimental multidimension plugin for reading spatiotemporal sources like:

netCDF raster files following the Climate and Forecast (CF) convention
GriB edition 1
HDF version 4

Release artifacts have been deployed on the GeoSolutions maven repository, as well as on the OSGEO one.

Regards,
the GeoSolutions Team

GeoSolutions @ OSGIS 2011

2011-06-20T00:21:00.000-07:00

Dear All,
just a quick note to inform our followers that GeoSolutions' founders Simone Giannecchini and Alessio Fabiani will be present at OSGIS 2011 on Tuesday and Wednesday (21st and 22nd).

As per the agenda, Alessio Fabiani and Simone Giannecchini, will be giving an introductory workshop on GeoServer 2.1 on the 21st, moreover on the 22nd Simone Giannecchini will be talking about GeoServer 2.1 (here you can find the slides for the talk).
As usual, we will make the workshop available for download for free from our website shortly after the conference. This time we should also make the material available for download separately.

If you are around and you are interested in knowing more about the latest advances for GeoServer or if you are interested in knowing more about who we are and what we do, just look for us!

The GeoSolutions team,

GeoServer 2.1 presentation for OSGIS 2011

2011-06-13T00:27:00.000-07:00

GeoServer @ Osgis 2011

View more presentations from GeoSolutions

Interested in knowing more about the latest advances for GeoServer? Contact us!

The GeoSolutions team,

Job Offer @ GeoSolutions

2011-06-02T01:30:00.000-07:00

GeoSolutions is looking for talented software engineers to fill a Junior GeoServer Developer position which would involve:

designing and implementing new GeoServer features
performing day-by-day bug fixing
mantainance of existing installations.

The candidate will be based in Rome (at least for the first 6 to 12 months), working on-site at the HQ of one of our largest clients.
Required qualifications are as follows:

Working knowledge of Java (JEE and JSE)
Working knowledge of GeoServer
Working knowledge of OpenLayers is a plus
At least 1 year of experience
Being fluent in English, both written and spoken

We offer a variety of contracts but, please, notice that our intention is to establish a long term relationship, therefore this is not a position for freelance consultants.

Working remotely is an option, although we will give priority to candidate closer to our office and/or close to Rome where the candidate will be initially based.

Please send a detailed resume together with a letter of presentation at jobs_at_geo-solutions.it.

The GeoSolutions team,

Job Offer @ GeoSolutions

2011-06-01T02:24:00.000-07:00

GeoSolutions è alla ricerca di software engineer di talento per coprire una posizione come Junior GeoServer Developer. Le mansioni sono le seguenti

designing and implementing new GeoServer features
performing day-by-day bug fixing
mantainance of existing installations.

Il candidato lavorerà a Roma (almeno per i primi 6 - 12 mesi) presso la sede di uno dei nostri clienti principali. E' comunque previsto un eventuale periodo di formazione iniziale, qualora fosse ritenuto necessario.

Le qualifiche richieste sono le seguenti:

Working knowledge of Java (JEE and JSE)
Working knowledge of GeoServer
Working knowledge of OpenLayers is a plus
Almeno un anno di esperienza nel settore specifico e con le tecnologie indicate
Ottimo inglese scritto e parlato. La lingua ufficiale parlata nella sede del cliente è l'inglese

L'offerta contrattuale è legata alla esperienza del candidato ed alla eventuale necessità di trasloco. Riteniamo comunque opportuno sottolineare come il nostro intento sia quello di creare un rapporto duraturo di collaborazione che si estenda oltre la opportunità specifica presso il cliente in questione.

Sara' data priorità a candidati che risiedono vicino alla sede del cliente o vicino alla nostra sede. Comunque sia chiediamo sin da subito disponibilità a brevi trasferte anche all'estero e comunque a trasferirsi vicino alla nostra sede in caso sia ritenuto necessario.

Chiunque fosse interessato è pregato di inviare un CV aggiornato, accompagnato se possibile da lettera di presentazione, a jobs_at_geo-solutions.it.

The GeoSolutions team,

GeoNetwork in da cloud!

2011-05-30T01:49:00.000-07:00

GeoSolutions is pleased to announce that GeoNetwork is now available on the Standing Cloud infrastructure.

Standing Cloud provides services that make it quick and easy to run open source applications in the cloud allowing users to focus on putting applications rather than worrying about the servers that run them.

Standing Cloud provides point-and-click application deployment and management on your cloud of choice with instant access to hosted applications. Focus on portability ensures no providers lock-in, so moving applications to a different cloud provider is a matter of a single click, at any time, for any reason.

Between available cloud partners, we have:

Amazon Web Services
GoGrid
Linode
Slicehost
Rackspace Cloud

GeoSolutions has joined the Partner Program of Standing Cloud and it is now listed as Solution Partner for GeoServer and will soon be listed for GeoNetwork as well. We have worked with the Standing Cloud team to integrate GeoNetwork 2.6.3 in their application network, so that now people can install and deploy a GeoNetwork instance in less than 15 minutes with no system administration skills!

Just look at the screenshots below, in 10 minutes with a few clicks I have gone from nothing to having GeoNetwork running.

So log in and have fun with GeoNetwork in da cloud!

If you have questions about the work described in this post, or if you want to know more about how our services could help your organization reaching its goals, do not hesitate to contact us

The GeoSolutions team

Developer's Corner: Turning GeoServer into an ImageServer

2011-05-23T00:13:00.000-07:00

Dear All,
we would like to briefly talk about some of the lastest developments we have performed at GeoSolutions in order to turn GeoServer into an ImageServer.

So far GeoServer has not been able to publish single non georeferenced images as WMS layers, it always required a valid EPSG for specifying a well-known geospatial coordinate reference system. Lately, we have been working with OpenGeo on an application whose goal was to allow users to view non-georeferenced images through a WebGis based on OpenLayers. We decided to take the risk to modify the GeoServer and allow it to serve pure imagery via the WMS protocol. First thing we did was to define an EPSG code with a valid identifier that could convey enough information to have clients understand we were serving a pure image. The debate on the GeoTools mailing list was long but helpful, see the following links for reference:

Long story short, we followed Jody Garnett's suggestion to define an EPSG identifier using the code 404000 for a 2D wildcard coordinate reference system in meter as shown here. For those who know what a WKT is, we are showing it here below

LOCAL_CS["Wildcard 2D cartesian plane in metric unit",
LOCAL_DATUM["Unknown", 0],
UNIT["m", 1.0],
AXIS["x", EAST],
AXIS["y", NORTH],
AUTHORITY["EPSG","404000"]]

If you are curios, here below you can see an simple non-georeferenced Jpeg shown in OpenLayers:

If you want to play with a live demo (we do not guarantee it will be up forever though...) you can follow this link.

If you want to know more about this development or if you are interested in knowing how GeoSolutions can help your organization to reach its goals, feel free to contact us.

Regards,
the GeoSolutions Team.

GeoServer upgraded to 2.0.3 release on Standing Cloud!

2011-05-17T06:18:00.000-07:00

GeoSolutions is pleased to announce that GeoServer has been upgraded from 2.0.2 to 2.0.3 (check the announcement for 2.0.3) on the Standing Cloud infrastructure. Notice that we have GeoServer 2.1 series on the radar for the next round of upgrades.

Standing Cloud provides simple, powerful, services that make it fast and easy to run open source applications in the cloud allowingusers to focus on putting applications at work in the cloud rather than worrying about the servers that run them. Standing Cloud makes it point-and-click easy to deploy and manage applications on your cloud of choice since it provides instant access to host applications on many cloud providers. Focus on portability ensures no providers lock-in which means that moving applications to a different cloud provider with a single click, at any time, for any reason, is feasible.

Between available cloud partners, we have:

Amazon Web Services
GoGrid
Linode
Slicehost
Rackspace Cloud

GeoSolutions has joined the Partner Program of Standing Cloud and it is now listed as Solution Partner for GeoServer. We have worked with the Standing Cloud team to integrate GeoServer 2.0.3 in their application network, so that now people can install and deploy a GeoServer instance in less than 15 minutes with no system administration skills!

Just look at the screenshots below, in 10 minutes with a few clicks I have gone from nothing to having GeoServer running:

So log in and have fun with GeoServer in da cloud!

The GeoSolutions team

Developer's Corner: GeoServer-Manager, super-simple Java client library for GeoServer

2011-05-11T00:25:00.000-07:00

Dear All,
we just wanted to spend a few words about a Java library we have just released under the MIT license for talking to the GeoServer REST interface. We have named this library GeoServer-Manager (yeah, a lot of brainstorming behind that name, really...).

Basically the goal of this library is to provide a simple yet usable way to interface a Java application with the GeoServer that would actually ease the work of the developers hiding away from them the relative complexity of the GeoServer REST interface without introducing a complex communication layer.

Here you have some basic facts about the library:

minimum set of external dependencies, namely apache-http-common and jdom
dead-easy API, see examples here
works with GeoServer 2.1 and 2.0.X
allows to interact with most common formats:

geotiff
shapefile
postgis
mosaic

GeoServer-Manager is open source and licenced under the MIT License. This library draws its essence from the efforts endured inside GeoSolutions in various projects. Partial funding has been provided by FAO.

Our intent is to keep this library lean and mean, therefore we do not intend to add complex stuff like a de/deserializable entity model into it, since we want to use it as a simple yet powerful communication layer with the GeoServer REST interface. However, we have on the radar the possibility to reuse this library in order to build a web service to provide means to interact with the GeoServer internal configuration via other protocols like SOAP or Spring Remoting.

If you have questions or if you just want to talk to us about the using the library in your project, please, subscribe to the mailing list here. In any case, do not hesitate to contact us.

Regards,
the GeoSolutions Team.

Preview: GeoRepository, advanced authorization manager for GeoServer

2011-05-04T08:44:00.000-07:00

Dear All,
in this post we would like to introduce an application we have developer here at GeoSolutions in order to cope with the requirements coming from enterprise deployments in terms of Authentication and Authorization for GeoServer; its name is GeoRepository.

Whoever has been facing the technical problems involved with deploying GeoServer inside an enterprise infrastructure following the SOA paradigm knows that in such context Authorization and Authentication capabilities are usually provided by external services in order to centralize such feature, fostering reuse of existing capabilities, minimize risk of duplication of functionalities as well as to avoid duplication of users' credentials. Moreover in such context Single Sign On (SSO) services are usually a must-have.

GeoServer comes with an integrated security subsystem that is able to handle both Authentication and Authorization which is quite powerful and customizable, however we had the need to extend it further in order to be able to provide fine grained control over users' authorizations as well as in order to better integrate with existing A&A services.

A typical and yet simple usage scenario for GeoRepository is depicted here below.

The possible types of authorizations that can be applied using GeoRepository are controlled via generic rules with priorities , are shown below.

Each rule comprises of:

User
Profile
Instance (since GeoRepository can control more than one instance of GeoServer, the typical enterprise scenario having a cluster of machines serving geospatial requests)
Service and Request which can be used to control grants on OGC services
Workspace which can be used to control grants on GeoServer workspaces
Layer which can be used to control grants on GeoServer layers
Grant
- ALLOW grants access if the rule is satisfied, DENY denies access if the rule is satisfied
- LIMIT applies only when we have spatial restrictions available, in this case if the ruse is satisfied we basically perform an allow with spatial restrictions.

For ALLOW and LIMIT grants we can, under some conditions, set also additional limits which are called details.

Available detail levels are:

Layer Details allows us to specify refined rules for a specific GeoServer layer. As an instance it is possible to override the default style or to filter the available styles' list. It is also possible to specify CQL filters for reading and/or writing (when possible) GeoServer layers' attributes.
Layer Attributes Details allows us to limit access to vector layer attributes. By default all attributes are readable/writable, but with this capability we can hide some of them or make them read only.

The possibilities introduced by having GeoRepository work right next to GeoServer are quite a lot, let us list some of them:

Fine grain control over which services, layers, workspaces, operations (of a service) a user can access
Fine grain control over which geospatial area for a layer (being it raster or vector) a user can access
Fine grain control over the WMS style that a use can apply for a layer with the possibility to perform style substitution on a per request basis depending on the credentials received
Fine grain control over which attribute (for vector layers) or bands (for raster layers) a user can access (this way it is possible to hide/show sensible attributes depending on credentials)

We should set up an inline demo soon, meanwhile, if you want to know more about the features we currently support or about our roadmap, please, do not hesitate to contact us.

Regards,
the GeoSolutions Team

ImageI/O-Ext 1.1-RC1 Released

2011-04-19T02:26:00.000-07:00

Dear all,
GeoSolutions is pleased to announce the ImageI/O-Ext 1.1-RC1 release which has been incubating for a long time.
Changes with respect to 1.0.x series can be summarised as follows:

Out-of-the-box Support for GDAL 1.7.3, which means no more patches are needed for GDAL Java bindings in order to access it from ImageI/O-Ext.
BigTiff support, breaking the 4GB TIFF limit.
EnhancedImageReadParam support. It extends the standard ImageReadParam by implementing Cloneable (used when supporting MultiThreading read operation) and a new DestinationRegion param to support oversampling/subsampling without specifying sourceSubSampling parameters. This may be used when dealing with readers which internally take care of performing subSampling/overSampling, such as the GDALImageReader
Experimental multidimension plugin for reading spatiotemporal sources like:

netCDF raster files following the Climate and Forecast (CF) convention
GriB edition 1
HDF version 4

Release artifacts have been deployed on the GeoSolutions maven repository, as well as on the OSGEO one.

Regards,
the GeoSolutions Team

GeoServer in da cloud!

2011-03-27T22:56:00.000-07:00

GeoSolutions is pleased to announce that GeoServer is now available on the Standing Cloud infrastructure.

Standing Cloud provides simple, powerful, services that make it fast and easy to run open source applications in the cloud allowingusers to focus on putting applications at work in the cloud rather than worrying about the servers that run them. Standing Cloud makes it point-and-click easy to deploy and manage applications on your cloud of choice since it provides instant access to host applications on many cloud providers. Focus on portability ensures no providers lock-in which means that moving applications to a different cloud provider with a single click, at any time, for any reason, is feasible.

Between available cloud partners, we have:

Amazon Web Services
GoGrid
Linode
Slicehost
Rackspace Cloud

GeoSolutions has joined the Partner Program of Standing Cloud and it is now listed as Solution Partner for GeoServer. We have worked with the Standing Cloud team to integrate GeoServer 2.0.2 in their application network, so that now people can install and deploy a GeoServer instance in less than 15 minutes with no system administration skills!

Just look at the screenshots below, in 10 minutes with a few clicks I have gone from nothing to having GeoServer running:

So log in and have fun with GeoServer in da cloud!

The GeoSolutions team

Developer's corner: adding two global equal area projections to GeoTools/GeoServer

2011-03-07T09:40:00.000-08:00

Recently we had the pleasure to implement two new map projections for GeoTools/Geoserver: the Eckert IV projection, and the Mollweide projection.

Both projections happen to be equal area projections.
These projections are commonly used in atlases to represent the whole world, as they don't alter the relative proportions of countries and continents.

This is actually an important feature missing from the common Google/Bing/Yahoo maps. They are all using a close relative of the Mercator projection, which greatly exaggerates the size of all countries at high latitude (for example, Greenland and Antarctica look a lot bigger than they actually are).

Compare that with Eckert IV, obtained calling GeoServer with the map reflector:
http://localhost:8080/geoserver/wms/reflect?layers=countries&srs=EPSG:54012

And here is Mollweide, obtained calling http://localhost:8080/geoserver/wms/reflect?layers=countries&srs=EPSG:54009

One final observation: unfortunately neither Ecker IV nor Molleweide have an official entry in the EPSG database.
In order to use them in GeoServer you'll have to edit your $GEOSERVER_DATA_DIR/user_projections/epgs.properties file and add the following two entries (mind, each definition has to be added as a single line, here we split them among multiple lines for readability sake):

54009=PROJCS["World_Mollweide",\
GEOGCS["WGS_1984",DATUM["WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],
PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Mollweide"],
PARAMETER["Central_Meridian",0.0],UNIT["Meter",1.0]]
54012=PROJCS["World_Eckert_IV",
GEOGCS["WGS_1984",DATUM["WGS_1984",SPHEROID["WGS_1984",6378137.0,298.257223563]],
PRIMEM["Greenwich",0.0],UNIT["Degree",0.0174532925199433]],PROJECTION["Eckert_IV"],
PARAMETER["Central_Meridian",0.0],UNIT["Meter",1.0]]

After that you can take your favorite world wide map and use the WMS reflector to get a quick map using the above projections.

These projections will be available in GeoServer 2.1 and GeoTools 2.7.
Interested in more map projections and datum shift improvements? Drop us a line!

The GeoSolutions team

Developers Corner: Support for TIME in GeoServer WMS

2011-02-18T07:05:00.000-08:00

Dear All,
at GeoSolutions lately we are working on improving support for the TIME dimension in GeoServer.

While the work is still ongoing we wanted to share with you a preview of what we are doing, therefore we created the video below.

Like this feature? Interested in funding this development? Drop us a line!

The GeoSolutions team

Presentazione su GeoServer da GeoSolutions al FOSS4G-IT 2011

2011-02-17T16:39:00.000-08:00

Foss4g it-2011 (italian)

View more presentations from GeoSolutions.

The GeoSolutions team

GeoSolutions Presentation on GeoServer at FOSS4G-IT 2011

2011-02-17T08:30:00.000-08:00

Foss4g it-2011 (english)

View more presentations from GeoSolutions.

A new year with GeoSolutions Enterprise Services

2011-02-14T10:23:00.000-08:00

GeoSolutions is proud to offer four levels of enterprise support services centered on the GeoServer platform to help organizations building enterprise-class Spatial Data Infrastructure (SDI), integrating and tuning best-of-breed Open Source geospatial frameworks into what we call the OpenSDI suite.

Thanks to its extensive experience in building and supporting enterprise class geospatial infrastructures, GeoSolutions will provide your organization with an outstanding level of support through the full project lifecycle for a successful and stress-free deployment.

If your organization was refraining from adopting Open Source software in the enterprise due to lack of professional support, we are here to help. Therefore it is now time to invest on features and support rather than on software licenses.

Our Support, Your Success!

More information can be found here. You can contact us here.

The GeoSolutions team

Un nuovo anno con i Servizi di Supporto Enterprise di GeoSolutions

2011-02-14T01:19:00.000-08:00

GeoSolutions è orgogliosa di offrire anche per il 2011 quattro piani di servizi enterprise incentrati sulla piattaforma GeoServer per aiutare i propri clienti a costruire una SDI (Spatial Data Infrastructure) di classe enterprise, attraverso l'integrazione e l'armonizzazione dei migliori framework geospaziali offerti dal mondo Open Source, la OpenSDI suite.

GeoSolutions grazie alla sua vasta esperienza nella costruzione e nel supporto di infrastrutture geospaziali di classe enterprise, fornirà alla vostra azienda uno straordinario livello di supporto durante tutto il ciclo di vita del progetto per ottenere con facilità un deployment ben riuscito!

Se la tua organizzazione era restia ad adottare soluzioni Open Source software per via della mancanza di supporto professionale, noi siamo qui per aiutarvi. E' arrivato il momento di investire in supporto e feature piuttosto che in licenze software.

Our support, Your Success!

Maggiori informazioni possono essere reperite qui. Ci potete contattare qui.

The GeoSolutions team

Presentazione su GeoServer al meeting italiano degli utenti GRASS e GFOSS

2011-02-10T04:00:00.000-08:00

GeoSolutions sarà presente domani al XII Italian GRASS and GFOSS users meeting con una presentazione su GeoServer. Ecco di seguito l'abstract:

GeoServer è un server GeoSpaziale Open Source scritto in Java, seguendo le comuni pratiche Java 2 Enterprise, per la gestione, disseminazione e analisi di dati geospaziali. GeoServer permette di distribuire, manipolare e analizzare i dati usando i più diffusi standard OGC (WMS, WFS, WCS, WPS), senza però dimenticare estensioni specifiche per una interazione trasparente con client come Google Earth e software commerciale in genere, ne gli ormai diffusi approcci basati su protocolli REST e GeoJSON per la distribuzione semplificata di semplici dati vettoriali.Obiettivo della presentazione è fornire allo spettatore una un’esaustiva panoramica dellefunzionalità offerte da GeoServer per la creazione di Spatial Data Infrastructure interoperabili, con particolare attenzione alle nuove funzionalità introdotte in GeoServer 2.1 e all’analisi di dati spaziali con il protocollo WPS 1.0.In chiusura verrano forniti esempi d’uso in casi reali di recente implementazione con analisi statistiche su dati raster e vettoriali con considerazioni relative a scalabilità e prestazioni delle stesse.

La presentazione sara' tenuta da Andrea Aime, lead developer del GeoServer.

Nel caso voleste seguire la presentazione in streaming video, trovate il link sulla home della conferenza. Attenzione, la presentazione si terrà alle ore 9:00.

The GeoSolutions team.

Developer corner: cross layer filtering coming down to GeoServer

2011-02-04T03:45:00.000-08:00

Ever needed to find the answer to a question involving the spatial relationship between two layers? Something like "get me all the coffe shops withing 100 meters of the subway M exits?" Or "show me all the industrial buildings in counties X, Y and Z"?

Both of those queries require finding features in one layer, possibly with an attribute filter, and then locate all the locate all the features in another layer that have some kind of spatial relationship with the first result set.
This is known as a spatial join and it's used to find object that do relate with each other by proximity.

Now, GeoServer does not natively support layer joining. Up to today doing a WFS request like the above would have entailed:

make a first WFS GetFeature request to get the locations of subway M
parse the result, build a MultiPoint geometry
use it in a second WFS GetFeature against the commercial activities layer using a DWithin filter against the above constructed geometries

If you are playing with a web based client that is troublesome, first there are two round trips to the server, second the web based client cannot normally deal with many points meaning it may well start consuming too much memory for practically dealing with requests that pull up large geometries (state boundaries, rivers and the like).

Having to do the same with WMS comes with the aggravation that the filtering is likely done with CQL, but a large geometry won't simply fit within the maximum length allowed for GET requests (can be worked around with dynamic SLD o POST requests, but that adds other complications).

Enter the new "querylayer" GeoServer module.
The module provides filter functions that query and summarize GeoServer layers data so that the whole process can be done in a WFS single request or a single, compact CQL filter.

Let's make a simple example using the GeoServer sample data: we want to find all bugsites inside the restricted area whose "cat" attribute values 3.
The wfs request would be:

Whilst a WMS request doing the same would use a CQL filter as simple as:

INTERSECTS(the_geom, querySingle('restricted', 'the_geom','cat = 3'))

The following map shows the result with some context:

Want to find all the bug sites within 200 meters from any road. Here we go:

and the CQL equivalent would be:

DWITHIN(the_geom, collectGeometries(queryCollection('roads','the_geom','INCLUDE')), 200, meters)

which results in the following map:

Pretty cool eh? Naturally all roses have their thorns and this is no exception:

The requests, while valid, are going to work only with GeoServer, they are not portable towards other servers
The machinery is not actually able to do an in database join, it just loads the data in memory first and replaces the function call with the result so that intersects/dwithin can hit the database directly. It means the intermediate data is memory bound, and we actually have some admin configurable limits to make sure the requests are not loading in memory too much data (when the threshold is crossed an exception will the thrown)

The "querylayers" module is now a GeoServer community module that can be downloaded as part of the trunk nightly builds (starting tomorrow). It is available for both GeoServer trunk and GeoServer 2.1.x, though in the latter case you'll want to add the -Dorg.geotools.filter.function.simplify=true JVM option to get the best performance out of the module (it enables a specific querying optimization that is available by default on trunk).

We'd like to thank GeoSmart for sponsoring the development of this module, it's a pretty useful addition to the GeoServer data access arsenal.

As said, joining support could be improved even further. Interested? Let us know!

The GeoSolutions team

Developers Corner: Improvements to the rendering engine on upcoming GeoServer 2.1/GeoTools 2.7

2011-02-04T03:26:00.000-08:00

In this installment we are going to look into the rendering speed improvements featured in the the upcoming GeoServer 2.1/GeoTools 2.7.0 release.

We recently talked about a very significant one already: raster reprojection performance has skyrocketed, providing an overall speedup factor of four to six on the overall reprojected WMS request chain (if you want to know about the pure GeoTools reprojection operation speedup, it's more alike a hundred times faster).

What you might not know is that the above is coupled by another speedup that we made for the WMS shootout 2010: the "direct rendering path", which makes GeoServer into a proper image server.
What's the difference between a generic WMS server and a image server you might wonder? Well, a generic image server takes all the data sources and paints them on top of a rendering surface, applying styling as it goes. The rendering surface is commonly a blank RBG(A) image in memory on which features and rasters and combined.
Now, the thing is, if you are starting with a raster, you already have an image: there is no need to go through the generic rendering machinery, JAI can do all the required transformations directly and more efficiently than Java2D.
GeoServer 2.1 calls the above the "direct rendering path", and it's activated every time a WMS request contains a single raster layer: this gives another four-fold speedup at high load, and 20/50% speedup at the single user level.

The goodies embedded in GeoServer 2.1 do not stop here, significant improvements have been done in vector data rendering as well:

The renderer fetches vector data and renders it in two separate threads, allowing to distribute the request load among two cores
Large polygons are isolines are now clipped to the rendering area before being passed down to the Java2D rasterizer, greatly improving the performance on larger isoline layers and avoiding issues with one of the oldest unresolved java2D bugs which might bring a server to his knees when a large isoline or polygon is painted with a dash array and the user starts zooming in a lot (so that only a small part of the geometry is in the viewing area)
A special map is setup to track of very small geometries, this helps when rendering a very large amount of small objects at a low zoom level as it avoids painting over and over in the same pixel (this trick was available on 2.0.x as well, but only on shapefiles)
Shapefile indexing of very large data sets has improved significantly, making for smaller, but at the same time more selective, spatial index files. If you are upgrading from 2.0.x you should remove all your .qix files and let GeoServer rebuild them to take advantage of the improvements

We hope to setup a benchmarking session allowing us to get fresh comparisons between 2.0.2 and 2.1.0 once the release is done, possibly using the WMS 2010 shootout workload: stay tuned for more details on this one.

Of course there is still a lot that can be done to improve rendering performance. Do you have a specific case that you want to be improved? Interested? Let us know!

The GeoSolutions team

GeoSolutions YouTube channel

2011-02-02T23:37:00.000-08:00

Hi all,
at GeoSolutions we decided to create a YouTube channel in order to have common repository for the videos that we make sometimes to demonstrate and/or explain new functionalities for the software we develop.

You can reach the channel here.

Enjoy!
The GeoSolutions team.

Developers Corner: Improving GeoServer WMS Cascading Support

2011-02-01T23:37:00.000-08:00

Hi all,
The ability to server WMS layers from a remote WMS server through a Geoserver installation (a.k.a. Cascading WMS) has been one of the latest improvements in Geoserver world in last couple of months. You can read about it more in one of our previous blog, where the basics have been explained already.

GeoSolutions team has since been improving the existing state and introduced some new features. A few patches have been created to fix some important issues:

Enabling cascading layer to be included in layer groups (GEOS-4273)
Make GetFeatureInfo requests work again on cascading servers (GEOS-4301)
Make GetFeatureInfo requests work on older version of geoservers (GEOT-3383)
Include "image/png; mode=24bit" to the tentative transparent formats used in WMSLayer (Mapserver transparent png support - GEOT-3393)
Other smaller bug fixes (GEOT-3361, GEOS-4295, GEOS-4286).

Batch import
The main improvement GeoSolutions has been working on, it's been the possibility to perform batch import of layers from a remote WMS (GEOS-4296) through the user interface. This development allow users to try and publish all layer, or only a certain subset, all at once. After the WMS store has been configured, new link providing batch import is shown.

Selecting batch import takes us to a new page where multiple layers can be selected and configured with a single mouse click.

Clicking on “Import selected” button imports selected layers and prints out simple status report on imported layer. Notice that the Published Icon is update accordingly.

We have made a video showing how to use the batch import of WMS servers into Geoserver. You can download it from here.

Do not hesitate to try it out and stay tuned for more WMS Cascading improvements!

The GeoSolutions team.

A new home for the ImageIO-Ext project

2011-01-30T15:30:00.000-08:00

Dear All,
after a long wait a new home has been finally created by Oracle for the ImageIO-Ext project on the new Java.net infrastructure. You can reach it here.

For the records the Image I/O-Ext project extends the Java's ImageI/O framework capabilities by providing access to new raster formats directly or through the integration of other frameworks and libraries, like GDAL, JMagick (experimental) and Kakadu for JPEG2000.
The project is Open Source but commercial-friendly, part of the code is released under an LGPL license, some other parts (namely, derivative work from Oracle's imageio source code) are licensed under BSD license.

This work has been started with the Google Summer of Code in 2006 by Ing. Daniele Romagnoli mentored by Ing. Simone Giannecchini (GeoSolutions).

The GeoSolutions team.

Workshop su GeoServer al meeting degli utenti GRASS e GFOSS

2011-01-27T06:34:00.000-08:00

GeoSolutions sarà presente al XII Italian GRASS and GFOSS users meeting con un workshop su GeoServer.

Il workshop fornirà agli utenti una veloce introduzione al framework GeoServer, concentrandosi essenzialmente sui seguenti aspetti:

basic installation
basic data configuration and publishing
basic data exploitation

Nel case voleste partecipare al workshop, non dimenticate di registrarvi in quanto i posti disponibili per la conferenza sono limitati!

The GeoSolutions team.

GeoServer Workshop at Italian GRASS and GFOSS Users' Meeting

2011-01-26T10:37:00.000-08:00

GeoSolutions will hold a GeoServer workshop at the XII Italian GRASS and GFOSS users meeting, the annual gathering of the Italian FOSS4G crowd.

The workshop will provide a quick introduction to the GeoServer framework and will mainly focus on:

basic installation
basic data configuration and publishing
basic data exploitation

In case you are around and you want to attend the workshop, make sure to register for the conference since seats are limited!

The GeoSolutions team.

Developers Corner: have your SLD transform raster data into vectors on the fly

2011-01-23T23:43:00.000-08:00

Hi all,

in this post we'd like to share our most recent endeavor in dynamic data rendering within the GeoServer and GeoTools open source projects.

The problem

Suppose you have a set of scientific raster data sets, maybe they represent some sort of concentration, elevation, or maybe they represent wind, currents, or some other vector phenomena via two bands (one for magnitude, one for direction).

Now, you have lots of them and people want to display them in various ways. Raster with color scales are nice, but often you need to render them in other ways, such as contour lines, polygons catching all the pixels within certain ranges, or vector fields (think wind barbs).

Those are all raster to vector conversion processes that a WPS can take care of. However, suppose you also have a ton of those raster data, and that the raster classification parameters need to be dynamic, with a user providing, for example, the contour levels to extract.

Now you're facing a somewhat hard problem, in theory you would have to:

call the WPS with the given data
store the results somewhere
register that new layer as a published WMS layer, along with the proper style
update the viewer to add that layer
purge that temporary layer once the user is done or wants a different set of parameters to be applied in the transformations

To add icing on the cake, suppose your datasets are massive, so doing the WPS extraction at full resolution can take its dear time.... does not really sound like a situation one would like your server and client infrastructure to deal with.

The solution

Instead of doing all of the above work, wouldn't it be nice to just specify the transformation needed in the style sheet? That's exactly the road we decided to follow.
We've created and SLD extension allowing to pipe a process (yes, a WPS one) inside the SLD so that it can be dinamically updated. It looks like the following:

The above would call on the fly the contouring process and then render its result: no need to create and manage a new vector layer, the data is generated on the fly only when needed.
Here is how the result looks (using a style sheet just a bit more complex than the above one):

Chaining transformations we can also extract and display the value of the single pixels and show it as a label, as in the following example:

Alternatively you may want to extract the polygons containing all the cells in a certain data range, like in the following transformation:

The result, coloring each range in a different way, is:

Finally, we may want to extract as set of wind arrows starting from a raster having the horizontal and vertical components of a vector (u and v):

We're linking to the full SLD of this last one because it's quite the testament of SLD flexibility: the magnitude and direction of the arrow are computed on the fly by using filter functions (functions that are part of GeoTools/GeoServer, you may not find them in just any implementation).

One important bit here is that the raster to vector conversion are happening at the visualization resolution: this means you can have the transformation work against large datasets without heavy slowdowns, because it is going to happen only in the area you're looking at, and at the resolution that would have been used to draw the raster.
This makes it possible to get fast, on the fly operations that do not excessively slow down rendering.

This is yet another example of how processing capabilities can be integrated into GeoServer, and it's by no means the last. Also, there is still plenty that can be done to improve this kind of transformations, as well as new transformations to support mapping tools such as heatmaps. Interested? Let us know!

The GeoSolutions team

GeoSolutions is sponsoring FOSS4G 2011

2011-01-21T01:55:00.000-08:00

GeoSolutions is proud to announce its participation has a bronze sponsor to the Free and Open Source Software for Geospatial (FOSS4G) 2011 conference, being held September 12 - 16, 2011, in Denver, CO, USA.

The GeoSolutions team.

Developers Corner: Improving GeoTools/GeoServer raster reprojection performance

2011-01-10T01:15:00.000-08:00

Ciao a tutti,
we hope everybody had nice seasonal holidays and is back fresh and ready to move on with another year of activity.

In this blog we'd like to present some of the work we have been doing in recent time to improve GeoTools and GeoServer raster reprojection abilities.

Raster reprojection is a quite heavy process in which every single pixel of the original image has to be mapped into a new position in the target raster. Here is a visual representation of a small set of pixels, before and after the reprojection (in this case, from WGS84 to polar stereographic):

A point by point reprojection is obviously taking quite a toll, a screen sized image has easily well over one million pixels to reproject, much more than the number of vertices one can find in the common vector map. To make things worse, the transformation to be used often involves trigonometric functions and other complex mathematical tools that visibly slow down the calculation.

The reprojection algorithm used by GeoServer and GeoTools until now worked exactly like that, it used a very accurate but also very slow approach.

Now, if you think about it, the transformation applied to one pixel and the one applied to the next one are basically the same, the difference is going to be minimal.If we only had a small bunch of pixels we could compute a simple translation and apply it to them all.
Having more pixels we could try to approximate the overall transformation with a simple linear one, which in two dimensional space is known as an affine transformation.

If the area gets larger a single affine transformation would not be good enough anymore, but we could use many of them to fit different areas of the raster.

Have a look at the following one dimensional representation to get the gist of the idea (this is known as piecewise linear function):

The trick to apply this approach if finding out where and how much to subdivide the original area into discrete pieces that can use a single affine transform: to do so we devised a simple error estimation method that proved to work well with all common map projections.

So we start by evaluating the linear transformation error over the entire raster area, if it's larger than a certain target we split the area and evaluate again, and recurse this way until the approximation error has been put under control.

The optimized method proved to be quite effective, drastically reducing the time it takes to reproject a raster. We made some tests with GeoServer using the same data and the same reprojection test used in the FOSS4G 2010 WMS shootout measuring a six times speedup:

We also compared the images generated by the original and optimized method finding there was hardly any human noticeable difference, and very little difference that could be machine detected at all.

For all of you that want to dig into the nitty-gritty details, we have a full tech report that you can inspect.

There is still plenty that can be done to optimize further raster data reading and in memory transformations to reach even higher performance and scalability. Interested? Let us know!

The GeoSolutions team

Greetings from GeoSolutions' CEO

2010-12-23T02:56:00.000-08:00

Developers Corner: Improving GeoServer security

2010-12-20T04:01:00.000-08:00

Hi,

during the last few weeks we have been thinking a lot about improving the GeoServer security, making it at the same time easier to integrate with external authorization systems and making it more expressive in terms of what kind of security rules can be expressed.

As part of the work we have been re-evaluating how the security can be applied in an ever growing software like GeoServer without having to cripple the available service abilities or to have to run after the developers constantly adding new features to the system.

The evaluation resulted in a (long-ish) comparison of the possible models, including the all popular security proxy as well as the existing GeoServer implementation, and coming up with an approach that extends what we already have for improved overall security and improved pluggability with existing systems.

Read the full comparison and the proposed solution in this GeoSolutions tech report

Interested in advancing GeoServer security subsystem? Let us know!

The GeoSolutions team

Estimating time and space required to build a GeoWebCache on disk cache

2010-12-01T03:19:00.000-08:00

An outstanding issue when using GeoWebCache is finding out how much disk space a certain layer will use, on disk, once fully cached, and also how much time it will take to pre-seed the cache up until a certain layer.

The GeoWebCache site used to link to a Google Documents spreadsheet providing such an estimate, but the link is gone from the current documentation and, regardless, we found the spreadsheet to be less than accurate, basically ending up computing at the very least the seeding size of 1/4 of the entire world no matter how small the given area was.

Therefore we set out to create a better replacement that would actually count the tiles needed to seed a specific area. The result works for a cache in WGS:84 and is available for everybody to look at and use on Google Documents: GWC estimation - actual tile counting. The current setup compute the actual cache size for the entire Italy.

The document could probably use some more improvements, we're actually looking forward to hear from you about the results you're getting and possible improvements to the spreadsheet.
Let us know!

The GeoSolutions team

GeoServer Tips & Tricks: Auto alignement of labels within polygons

2010-11-29T01:19:00.000-08:00

Labeling polygons can sometimes be tricky. GeoServer ormally places the labels horizontally within the polygon, accepting only labels that "sit" inside the polygon for at least half of their length.
However sometimes the polygons just don't collaborate with that setup, they can be thin and long, but rotated on an non horizontal angle: in this case GeoServer would not label them until the user zoomed in quite a bit in the map.

Some time ago Björn Hartell and Andrea Aime started working on a patch to add a vendor option auto-rotate the labels for those polygons. After a few iterations and work on both sides we ended up with some pleasing results.
Here is a couple of examples from the well known countries map from Natural Earth labelled with the "minimum bounding rectangle" auto rotation option:

Labelling buildings or parcels is another common usage for this option:

The map was obtained using the following vendor options in the TextSymbolizer section of the SLD:

<VendorOption name="polygonAlign">mbr</VendorOption>
<VendorOption name="maxDisplacement">20</VendorOption>
<VendorOption name="goodnessOfFit">1.0</VendorOption>

The patch just landed in GeoTools, meaning it will be available in the next GeoServer 2.1 beta release.Notice that we have more cartographic rendering improvements in the pipe.

Interested in getting better maps on screen, having some hard rendering problem to tackle? Let us know!

The GeoSolutions team

Fun Stuff: Computing circular buffers in geographic coordinates

2010-11-24T09:48:00.000-08:00

Finding all the objects within a certain distance from a point is surely a common GIS problem. The problem is normally solved using OGC "dwithin" filters or by computing a buffer and then finding all the intersecting objects.

Very often both of the approaches fail miserably in case the coordinate system is a geographic one, as common libraries, such as JTS and GEOS, are not able to handle the non planar nature of it. As far as "dwithin" is concerned rencent Oracle and PostGIS versions can manage the problem properly, but what to do if they cannot be used?

We had to solve this problem when computing data distribution statistics over raster data cells that are within a certain distance from a given point, and making for an accurate calculation regardless of how long the distance was.

To do that we created a new GeoServer WPS process, "gs:PointBuffers", that can create a set of buffers given a point, a target SRS and a set of distances in meters.

In case the SRS denotes a geographic spatial reference system the GeoTools GeodeticCalculator is used to sample the set of points that are at the given distance, looping over a closed sets of azimuths to cover the entire shape.

Interested in seeing the results? I certainly was.

Let's start with a set of small buffers at a medium latitude: 10, 30, 50 and 100 km buffers around a point located in northern Italy. Here is there result:

As you can see, drawing the result in plain WGS84 (plate carré for the conoisseurs) we get elliptical shapes. This should not come as a surprise if you consider that at 45° one degree of latitude spans 111km, whilst a degree of longitude spans only 78km (see the "Degree length" table at Wikipedia).

What if we pump up the distance significantly? Let's try with 100, 500, 1000, 2000 and 3000km instead. Here is the result:

See the funny shape we get? This is the effect of the size of one degree of longitude shrinking as we move towards north.

It is also a good indicator of how deformed the now common WGS84 maps, often published on the web, are.

If you want to see the same data in a common projection, let's have a look at the same map in EPSG:3857 (aka the Google projection):

Somewhat better, even if the Mercator tendency to inflate areas at high latitudes is well evident.

Well, this is it. The gs:PointBuffers is soon going to land in GeoServer for your testing pleasure.

We'd very much like to tackle the same problem against lines and polygons as well. Interested? Let us know!

The GeoSolutions team

(CAS + LDAP + FTP + LifeRay) ^ GeoSolutions = Enteprise Portal

2010-11-23T02:36:00.000-08:00

Recently GeoSolutions has developed for one our biggest clients their new web portal . The long term goal was to rationalize and concentrate all users and applications management in a single entry point, providing them with all the functionalities of a classic web portal.

Use Case and Requirements
The development and deployment of the whole architecture was challenging due to the strict functional and non-functional requirements they put forward.
The use case is quite complex since our client periodically conducts experiments at sea, collaborating with other agencies and research centres, where large amounts of data are collected and stored. Moreover, security is a crucial concern for it and therefore sophisticated and fine grained access control and authorization enforcements mechanisms are required.

Main requirements were as follows:

introduction of a Single Sign On (SSO) mechanism in order to allow users to share their credentials across all the applications exposed through the web portal
the system had to be able to distinguish external users from internal users and therefore present a different set of security checks as well as richer or poorer content according to their provenance.
usage of a single interoperable and open credentials respository for storing users credentials accounting also for the possibility to use them also for FTP A&A as well as for legacy applications.
fine grained RBAC control capabilities in order to group users according to their needs as well as their provenance

Let us briefly explain the concept and the work done in order to implement such a system.

Single Sign On
We decided to implement the SSO system trough a Central Authentication Service (CAS).
The CAS Service assigns authentication tickets to the users after a successful logon throught HTTP cookies so that an application or a service (like Apache HTTPD for instance) able to communicate with CAS is able to automatically recognize the user and its authentication status.

Authorization
Notice that with CAS we can provide only Authentication, not Authorization. That means that we need some other mechanism able to manage user's authorizations, the choice we made was to use LDAP to implement Role Based Access Control (RBAC).
RBAC was accomplished by creating a specific LDAP users repository and enabling both HTTP and FTP daemons to draw users privileges from it. The LDAP service was provided by an OpenLDAP server with a custom schema for groups and peoples.

The Web Portal
The web portal was implemented through LifeRay. LifeRay was configured to provide authentication through CAS and authorization through LDAP. In other words it imports/exports users and groups from/to the LDAP server. All the others A&A LifeRay mechanisms have been configured accordingly.

Putting it all together
Here below a schema of the entire system and its components.

Let us now explain in more details the architecture above.

Apache HTTP Server sits in front of every direct HTTP access to data configured with “mod_cas” and “mod_ldap” in order to provide A&A. The “mod_rewrite” and “mod_proxy” modules have been also configured in order to both recognize internal/external accesses and forward requests to internal web apps.

Both CAS and FTP services have been configured to get users credentials from LDAP.

In order to provide the customer with a fully functional system allowing him to manage credentials and web applications areas, we also developed a specific portlet for allowing administrators to manage users' credentials in LDAP as well as to provide users with the capability to reset their password.

Final Notes and next steps
We had to customized LifeRay deeply in order to integrate all the above components, but thanks to its flexibility as well as to its extension frameworks we have been able to accomplished what we needed to without touching the source code nor recompiling the standard distribution.
Next steps includes integrating also the GeoServer into this infrastructure and letting it play nice with LDAP and SSO. Stay tuned for more info on the topic.

The GeoSolutions team.

GeoServer workshop available for free!

2010-11-22T13:16:00.000-08:00

Dear All,
a quick post to just let you know that GeoSolutions is sharing an introductory workshop on GeoServer 2.0.

The material comprises of:

a light Ubuntu-based ISO
a preconfigured GeoServer data directory containing data kindly donated by the Municipality of Prato in Italy
the workshop itself, as a set of html files

The ISO file can be run either using Virtual Box or the VMWare Player. As an alternative you can burn it on a CD and you can use it as a Live Ubuntu or install it on your system.

Once you get it up and running, the installation contains:

An instance of the GeoServer 2.0.x stable nightly with a subset of the Prato's layers with proper styling.
PostgreSQL with Postgis
UDIG
QGis

The topics covered in this workshop are as follows:

Installing and running GeoServer: describes how to install and run GeoServer

Adding Data to GeoServer: explains how to manage the GeoServer Data Directory as well as how to add the base data types into GeoServer, like ShapeFile, GeoTIFF as well as PostGIS data

Pretty maps with GeoServer: describes how to manage the GeoServer maps visualizzation. All aspects related to styles, layer groups and other interesting GeoServer features affecting the WMS protocol will be introduced

Google Earth and Maps support: introduces GeoServer support for KML and Google Earth. The reader will learn how to view GeoServer data in Google Earth, and will be exposed to some of the more advanced features of KML output.

You can download the workshop from here:

In case you find errors or issues, or in case you want to know more about our services with regards to training, please contact us!

The GeoSolutions team,

Presentation on GeoServer @ Gfoss Day 2010

2010-11-16T04:09:00.001-08:00

Dear All,
a quick blog post to share the presentation Simone Giannecchini is going to give at the Italian GFOSS Day 2010 tomorrow morning about GeoServer.

Gfoss 2010

View more presentations from GeoSolutions.

The GeoSolutions team,

GeoSolutions helps GeoServer WPS going mainstream

2010-11-15T01:23:00.000-08:00

GeoSolutions is funding Andrea Aime's time to bring the GeoServer Web Processing Service (WPS) module to become an officially supported extension. A proposal has been submitted and is being voted on by the Project Steering Committe; you can track the progress here.

For those who have no idea of what WPS means I can cite part of the description I have found in wikipedia, which is very informative:
"The (WPS) is designed to standardize the way that GIS calculations are made available to the Internet. WPS can describe any calculation (i.e. process) including all of its inputs and outputs, and trigger its execution as a Web Service. WPS supports simultaneous exposure of processes via HTTP GET, HTTP POST, and SOAP, thus allowing the client to choose the most appropriate interface mechanism. The specific processes served up by a WPS implementation are defined by the owner of that implementation. Although WPS was designed to work with spatially referenced data, it can be used with any kind of data.
WPS makes it possible to publish, find, and bind to processes in a standardized and thus interoperable fashion. Theoretically it is transport/platform neutral (like SOAP), but in practice it has only been specified for HTTP.
WPS defines three operations:

GetCapabilities returns service-level metadata
DescribeProcess returns a description of a process including its inputs and outputs
Execute returns the output(s) of a process"

GeoSolutions plans for the WPS module in GeoServer extend beyond simply making it an official extension; in the short term we intend to publish a set of processes that we have developed for FAO which will add support for sophisticated statistics on both raster and vector data as well as other more specific processes (raster crop by polygon to name one).

In the longer term our goals include the following items:

Exposing GDAL utilities (e.g. gdalinfo) as WPS processes
Exposing Octave functions as WPS processes
Exposing IDL routines as WPS processes
Raster Algebra support via JAI-Tools
Improving support for grid processing
Improving support for clustered processing for superior scalability

Stay tuned for more information and, if you want to know more, you can always contact us directly!

The GeoSolutions team.

SimpleFeatureService: yet another attempt to simplify vector data dissemination - Part I

2010-11-11T04:21:00.000-08:00

In the current web gis world the most common standard protocol to disseminate vector data is the OGC Web Feature Service (WFS). While the protocol is fine, it has a number of shortcomings if we focus on the simple features dissemination case:

the protocol is complex, writing a client is often hard and error prone, same goes for implementing a new WFS server
the axis order issue makes it difficult, even today, to know what the axis order of the data returned by a WFS is
usage of GML implies there are many ways to present the same information or the same geometry (think polygon vs surfaces, or the issue that sometime arise with the feature built in attributes, such as name and location, with the ones that the application wants to publish)

In SOA architectures it is common to have different pieces of the overall system written in different languages, and sometimes there is a need for some part to exchange simple vector data information: we were looking for a quick to implement protocol that would get the job done without the extra complications of WFS (btw, we're not throwing away standards, in fact GeoServer WFS would be used to cascade the simpler protocol into the more complex, but standard compliant, WFS 1.1).

A quick research showed that GeoJSON would have been a very good candidate for data transport, since there are ready made parsers and encoders for the most common languages. The existing Mapfish data query protocol also seemed to fit the bill quite nicely, however we found some limitations that make it hard to use the protocol as a source of data to be cascaded via WFS:

there is no entry point that lists all the available layers
there is no way to know in advance what attributes will be returned and what is the spatial reference system of the geometries, nor is the axis order known (we want that to be explicit, just to make sure we avoid the same issues we're seeing in WFS)
there is no way to quickly get just the bounds and count of the features satisfying a certain filter
there is no recognized way to eventually transport vendor specific parameters (in our case, a multidimensional query for OLAP systems)
the pure restful-ness of the protocol does not allow to pass over many parameters

What we did was to extend the protocol a bit to support the above requirements, making it into a very simple OGC like service, and making concessions to the usage of POST requests to gather data.

Well, we can already hear the restful purists yelling about the usage of POST requests to get data. We're aware that it would have been possible to handle that, but each solution would have had drawbacks:

storing the query as a resource would have required to add urls to handle such resources, and some way to expire them (can't trust the client to orderly go and remove them manually)
providing a link back to the full query would have implied the client can publish resource and that the server can link back to them, both cases are not always met in practice

The focus of this protocol is implementation simplicity, so that a diversity of implementors in different languages can quickly get a simple feature service up and running, so we believe that in this case pragmatism needs to have the upper hand over religion. If it's possible to implement the service quickly and without ambiguities we basically reached our goal, any other improvement can be of interest provided it does not hamper this primary objective.

A PDF document describing the current layout of the simple feature service procotol is available. There are a few questions that we feel are still open:

the layout of the urls could probably follow a more hierarchical structure
the data type conventions could be fleshed out in more detail (thinking about date/time/timestamp for example)
the feature attributes is an unordered set, however cascading by WFS will result in some order being chosen. Wondering if the order should be established at the simple feature service protocol directly (by using an array of attributes in the describe layer output)
the documents may be linking each other (this would be make the service crawlable by search engines, assuming that is even a goal and not, on the contrary, an undesirable feature)
the capabilities could be turned into a simple list of names and we could have the describe resource provide also the spatial reference system and bounds informations

Let us know what you think!

Regards,
the GeoSolutions Team

WorldFile explained (or at least we tried!)

2010-11-08T07:46:00.000-08:00

There is a lot of confusion around about what world file represents therefore we thought it would have been nice to put together a couple of clarifications.

The World File
World File is a de-facto standard from early days of ESRI desktop GIS applications. Originally it was composed by at least two files, a raster file, let' say a file.tiff, and another file (most part of the times called file.wld) that contains numbers that define the affine transformation between the image and the target spatial reference system. This implicitly means that we are talking about georectified raster data. This wld file is what should be properly called the world file. Other extensions that could be used on behalf of tfw are wld tifw, etc... Moreover some other formats beside tif can be used, like gif, png, jpeg, jpeg2000, etc..and the extensions for the world file become pretty fancy (gifw, pgw, etc..).

The concept behind this couple is as follows. An image (e.g. a tif) plus its world file represent a geospatial georectified raster in some spatial coordinate reference system (CRS). The mapping between the raster space (the intrinsic Image space of rows and cols) and the associated model space is represented by an affine transformation whose coeffficients are contained inside the accompanying world file (see here ). This way, practically any image format can be made spatial-aware just adding a world file containing an affine transform that maps image points onto a crs. Here is a good explanation of what a world file contains.

The Projection File
One would now ask, ok, but what coordinate reference system I am supposed to use. The best answer would be "the one the person who produces the image + the world file was using"! Seriously many people make assumptions on the CRS used for the georeferencing affine transform contained in the world file, but in fact this can lead to all sort of problems. The best way to transmit the information about the spatial coordinate reference system is providing a third file with extension prj which contains the well known text (WKT more info can be found here) representation of the user spatial CRS. The Spatial Reference website can be extremely useful for associating the proper WKT to a certain CRS.

Recommendation
Notice that usually the raster space has x going right and y going down while in model space things DEPEND on the spatial CRS, specifically, axes order and axes directions!
ALWAYS ASK YOUR DATA PROVIDER TO DESCRIBE EXACTLY THE SPATIAL COORDINATE REFERENCE SYSTEM THAT WAS USED TO GEOREFERENCE YOUR DATA.

The GeoSolutions Team,

GeoSolutions ad ASITA 2010

2010-11-08T00:30:00.000-08:00

GeoSolutions sara' presente nella giornata di mercoledi 10 Novembre 2010 alla 14esima conferenza nazionale ASITA che si terrà a Brescia dal 9 al 12 Novembre 2010. Simone Giannecchini, fondatore di GeoSolutions, sarà presente per visitare alcuni dei nostri clienti e sarà disponibile per tutto il giorno per eventuali approfondimenti sulle nostre attività. Chi volesse, puo' contattarci per esprimere il suo interesse qui.

Il team di GeoSolutions

GeoSolutions sponsorizza il GFOSS Day 2010

2010-11-05T00:40:00.000-07:00

GeoSolutions è orgogliosa di annunciare la sua partecipazione come sponsorSILVER alla terza conferenza italiana sul software geografico libero, GFOSS DAY 2010 che si terrà a Foligno (PG) il 18 e 19 Novembre 2010.

GeoSolutions proporrrà inoltre un workshop di due ore sul GeoServer.
Per iscriversi al workshop è necessario registrarsi alla conferenza in quanto la partecipazione al workshop è limitata dalla disponibilità di computer.

Il team di GeoSolutions

Workshop su GeoServer al GFOSS DAY 2010

2010-11-04T01:12:00.000-07:00

GeoSolutions will hold a 2 hours GeoServer workshop at GFOSS DAY 2010, the annual gathering of the Italian FOSS4G crowd.

The workshop will provide a quick introduction to the GeoServer framework and will mainly focus on:

basic installation
basic data configuration and publishing
basic data exploitation

In case you are around and you want to attend the workshop, make sure to register for the conference since seats are limited!

The GeoSolutions team.

A new file chooser for GeoServer

2010-11-03T03:42:00.001-07:00

Creating a new shapefile or directory store in GeoServer is nowadays quite easy, you basically just have to specify the path to the file or the directory and GeoServer figures out the rest. However, that still requires you to hand type the path to the file.

I normally don't mind when running GeoServer locally, but it sure gets annoying when GeoServer is deployed on a remote host and a ssh session is needed to pick up the exact paths.
However it's really doing GeoServer workshops that one sees poeople used to desktop software getting out of their comfort zone with this "type a path" thing.

So lately we've been developing a patch adding an interactive file chooser that allows you to pick a file or a directly but simple point and click. Let's have a look:

The new shapefile store dialog provides with the "browse" button that will open the file chooser:
The file chooser in action, picking up a shapefile:
Choosing a directory of shapefiles instead:
The result of picking up the directory. As you can see we've choosen a directory inside of the GeoServer data directory and a relative path has been created (ensuring the data directory configuration remains portable among machines)

If you want a more dynamic display ve've also made a quick screencast (apologies for the low quality of it) that you can download from the GeoSolutions demo server. If you are on Linux you should be able to display the OGG Vorbis format directly, on other platforms you can install VideoLan to play it.

This new functionality will soon land into a GeoServer 2.1.0 near to you :-) .

The GeoSolutions Team,

"Geoserver for the masses" from the Tanzanian GIS User Group

2010-11-01T09:41:00.000-07:00

Dear All,
just spreading the word on this nice article that evaluates/introduces GeoServer as a tools to easily disseminate geosaptial data. We are actually very proud to hear the following:

Although GeoServer will not find the same audience as Google Earth it helps by leveling the field when it comes to providing spatial data by the use of servers. Where one earlier would need detailed knowledge (and funding) to set up ESRI products, one may now do the same investing only a couple of hours of work. Within hours you could be able to present spatial data within your own organization, or even externally using a web server.

If you are interested in reading the full article, which also contains a small intro on how to publish and style data with GeoServer, you can find it here.

The GeoSolutions Team.

WMS cascading in latest GeoServer

2010-10-28T02:04:00.000-07:00

One of the new exciting features in the GeoServer 2.1 series is WMS cascading.

For those that are not familiar with it, cascading allows to expose layers coming from other WMS servers as if they were local layers. This provides for some interesting advantages:

Clients connecting to your SDI need to care about less points of origin, which might be important for high security networks
It is now possible to ask for maps in formats not supported by the original server, or to reproject the maps in projections not supported by the original server (GeoServer supports out of the box almost 5000 different coordinate reference systems)
It is now possible to mix the layers with local ones to generate print oriented formats such as PDF
It is now possible to provide more informations about the layer, such as a better description, more keywords, which will benefit all clients, in particular catalogs harvesting informations from your capabilites document

Configuration

The configuration as usage of the cascaded layers follows GeoServer traditional ease of use.

The first step is to configure a new WMS server to be cascaded. In GeoServer terms that means configuring a new store, a WMS store in particular.

The only parameter needed is the capabilities document of the cascaded server, in this case, the DEMIS one (http://www2.demis.nl/wms/wms.asp?wms=WorldMap&request=getcapabilities&version=1.0.7)

Past this point it's business as usual, one just need to choose which layers to cascade using the normal layer configuration workflow:

Result

Here is GeoServer map preview showing two layers, Topography and Borders:

Of course it is also possible to mix in some local vector layers, such as, for example, the Tasmania layer group:

As said, one of the nice things about cascading is that one can add a few features, such as reprojection, transparently. The original modis server does not advertise reprojection abilities, yet GeoServer can reproject the data into EPSG:3785 on the fly for us:

Missing Features
There is still a number of things that can be done vs cascading:

Support for secured remote servers
Support for vendor parameters in cascading, including custom styling
Better networking support, with HTTP 1.1 persistent connections, limitations on the number of parallel requests to the cascaded server, and configurable timeouts
Support for choosing the cascading format (right now PNG is used)
Support for GetFeatureInfo cascading on servers that do not provide GML output (if the GML GetFeatureInfo output is there full cascading is already in place and working!)
Cascading WMS 1.3 servers
Auto configuration of the layers exposed by a remote server, so that new layers are automatically exposed by GeoServer without any manual intervention

Are you interested in any of the above features? Let us know!

Regards,
the GeoSolutions team.