Postgres 9.2 has introduced JSON as a server data type. At this point, the data was simply stored on server side with integrated wrappers checking that data had a correct JSON format. It was a good first step in order to store directly JSON data on server side but core features in 9.2 have its limitations in terms of JSON data manipulation and transformation.

Two new sets of JSON features have been added to PostgreSQL 9.3 planned to be released this year: functions related to data generation and a new set of APIs for data processing. The one this post deals with the ability to generate JSON data based on existing data types. The second set of features (operators and new processing functions) will be explained in a future post.

So… Functions for JSON data generation have been added by this commit.
commit 38fb4d978c5bfc377ef979e2595e3472744a3b05
Author: Andrew Dunstan
Date: Sun Mar 10 17:35:36 2013 -0400
 
JSON generation improvements.
 
This adds the following:
 
 json_agg(anyrecord) -> json
 to_json(any) -> json
 hstore_to_json(hstore) -> json (also used as a cast)
 hstore_to_json_loose(hstore) -> json
 
The last provides heuristic treatment of numbers and booleans.
 
Also, in json generation, if any non-builtin type has a cast to json,
that function is used instead of the type's output function.
 
Andrew Dunstan, reviewed by Steve Singer.
Catalog version bumped.

The first function called to_json permits to return a given value as valid JSON.
postgres=# create table aa (a bool, b text);
CREATE TABLE
postgres=# INSERT INTO aa VALUES (true, 'Hello "Darling"');
INSERT 0 1
postgres=# INSERT INTO aa VALUES (false, NULL);
INSERT 0 1
postgres=# SELECT to_json(a) AS bool_json, to_json(b) AS txt_json FROM aa;
bool_json | txt_json
-----------+---------------------
true | "Hello \"Darling\""
false |
(2 rows)

Boolean values are returned as plain true/false, texts are quoted as valid JSON fields.

json_agg is a function that can transform a record into a JSON array.
postgres=# SELECT json_agg(aa) FROM aa;
json_agg
---------------------------------------
[{"a":true,"b":"Hello \"Darling\""}, +
{"a":false,"b":null}]
(1 row)

The other tools for data generation are included in the contrib module hstore. Do you remember? This module can be used to store key/value pairs in a single table column. It is now possible to cast hstore data as json with some native casting or with function hstore_to_json.
postgres=# CREATE TABLE aa (id int, txt hstore);
CREATE TABLE
postgres=# INSERT INTO aa VALUES (1, 'f1=>t, f2=>2, f3=>"Hi", f4=>NULL');
INSERT 0 1
postgres=# SELECT id, txt::json, hstore_to_json(txt) FROM aa;
id | txt | hstore_to_json
----+------------------------------------------------+------------------------------------------------
1 | {"f1": "t", "f2": "2", "f3": "Hi", "f4": null} | {"f1": "t", "f2": "2", "f3": "Hi", "f4": null}
(1 row)

Note that in this case boolean and numerical values are treated as plain text when casted.

hstore_to_json_loose can enforce the conversion of boolean and numerical values to a better format, like that:
postgres=# SELECT id, hstore_to_json_loose(txt) FROM aa;
id | hstore_to_json_loose
----+-----------------------------------------------
1 | {"f1": true, "f2": 2, "f3": "Hi", "f4": null}
(1 row)

And now boolean and integer values inserted previously have a better look, no?

Having such tools natively in Postgres core server is really a nice addition for data manipulation and transformation of values into legal JSON.
However, you need to know that this set of tools is only the top of the iceberg for the JSON features added in 9.3… There are also new operators and APIs, which will be covered in more details with examples in one of my next posts. So… TBC.

Redis is an open source project providing key/value store features in a database server. This means that basically you can store in the database a value that has a given key and then retrieve the value using its key.
It supports advanced data types like strings, lists (elements sorted by insertion order), sets (unordered collection of elements) and sorted sets (collection of elements ordered by a key given by user). There are also other things supported like hashes or atomic integer incrementation. Feel free to have a look at the documentation about that.

This post will not deal with advanced aspects of Redis: this will be the subject of other articles. What is going to be described here is how to get Redis, run a server and then perform some simple operations.

Redis is honestly a famous project (5,000 followers on Github), so I am sure you will be able to install it easily whatever your environment.
For example, in the case of Archlinux, you only need to use a simple pacman command.
pacman -S redis
For CentOS:
yum install redis

But you honestly feel more what you do if you compile and install the code yourself from Github for example. So let’s get the code.
mkdir $REDIS_CODE
cd $REDIS_CODE
git init
git remote add origin https://github.com/antirez/redis.git
git fetch origin
git checkout unstable

The unstable branch is the main development branch, similar to master in the case of postgresql. You can refer to other branches for stable releases like 2.6 or 2.8.

The server code is located in folder src/, but you first need to compile the dependencies or you will get errors of the following type:
$ make
clang: error: no such file or directory: '../deps/hiredis/libhiredis.a'
clang: error: no such file or directory: '../deps/lua/src/liblua.a'
make[1]: *** [redis-server] Error 1
make: *** [all] Error 2

So here be sure to install first the dependencies as below.
cd deps
make lua hiredis linenoise

Then finalize compilation.
cd $REDIS_CODE/src
make

Finally install the binaries in a wanted folder.
make PREFIX=$REDIS_INSTALL install

Once installed, you will notice several binaries but the most important ones are redis-server (used to boot a server) and redis-cli (client to connect to a server).

In order to launch a server on default port 6379, simply launch this command, assuming that $REDIS_INSTALL is included in PATH:
redis-server

A Redis server can use a configuration file when booted, which can be specified like this:
redis-server /path/to/conf/redis.conf
There is also a template of redis.conf in the root tree of source code.

All the options of redis.conf can be specified via command line, here are some of them I find pretty useful for beginners.

  • –dir $DIR, to specify the directory where database dump file or log files are written to. The default value is “./”, so all the files are written in this case in the folder where redis-server is launched. I personally find that not really intuitive but…
  • –port $PORT_NUMBER, port number where server listens to. Default is 6379.
  • –logfile, name of file where logs are written. Default is stdout. Once again here I recommend using a clear file name combined with –dir to bring clarity to your database servers.

Then, in order to connect to the server, simply use redis-cli (see redis-cli –help for details about the options).
$ redis-cli
redis 127.0.0.1:6379>

Then you are ready to operate on your server. Let’s do here a simple get/set.
redis 127.0.0.1:6379> set foo bar
OK
redis 127.0.0.1:6379> get foo
"bar"

A last thing, I quickly wrote two scripts in case you are interested:

  • redis_compile, script that can be used to compile code, perform tests and do some other tricks
  • redis_start, script that can be used to set up a Redis cluster with master and slaves

Please note that those scripts do not have the granularity necessary for a use in production and they are only dedicated to development.

I have not yet discussed about the numerous features of Redis like things related to the cluster structure (master/slave replication, memory management), or data structures (lists, sets), but they will be covered in some future posts.

After obtaining a machine with MacOS Lion in it, here are the applications that look absolutely mandatory.
This list is more a memo than anything, but you could find it useful.

  • ClamXav, a free open-source antivirus. This can be downloaded from the AppleStore. Scans are light and quick.
  • LibreOffice, great free ressource for documentation, and has no Oracle logo included with it.
  • Xcode, a free development kit for Mac applications. It includes a gcc compiler, and can be found in the AppleStore.
  • iterm(2), alternative for terminal knowing that the default terminal in MacOS is a pain to use. iterm is not fully compatible with MacOSX Lion but iterm2 can be a solution.
  • vlc, a video viewer ressource. I’m with it for a couple of years, and it has always been astonishing light and fast.
  • MacPorts, an alternative way to manage packages with command line.
  • Minecraft, no comments on this one…

And that’s all…

©2010-2013 Michael Paquier All content is ©Copyright of Otacoo.com 2010-2013. Privacy Policy - Terms of Use