Postgres dev: create your own RPM packages

PostgreSQL No Responses »
Jun 182013

Creating RPM packages for your own needs can be quite an adventure the first time, especially with a code base as complex as Postgres considering all the submodules, options and extensions that the core code can come up with. But thanks to all the work done by the community, the building process is quite easy.

First, be sure to get the RPM specs from the official RPM repository.
git svn clone http://svn.pgrpms.org/repo/rpm/redhat

The spec repository has a structure depending on the version of postgresql, the Linux distribution on which the RPMs are based, and the package that needs to be built. There is also a common part called common/ where are defined all the rules used by all the Makefiles of each package, like some rules for calls as rpmbuild.

Depending on the version or the package you want to build, simply move into the dedicated folder:
cd $VERSION/$PACKAGE_NAME/$DISTRIBUTION
$VERSION should be something like 9.3, $PACKAGE_NAME named as postgresql, and $DISTRIBUTION be like EL-6 or F-17 depending on the Linux distribution where you want to build the packages.

Then run this command to create the packages.
make rpm
And you should basically be done… But actually not if your environment lacks some libraries.

You might run into dependency issues.
error: Failed build dependencies:
tcl-devel is needed by postgresql93-9.3beta1-4PGDG.rhel6.x86_64
e2fsprogs-devel is needed by postgresql93-9.3beta1-4PGDG.rhel6.x86_64
uuid-devel is needed by postgresql93-9.3beta1-4PGDG.rhel6.x86_64

The necessary libraries can be found with the flag BuildRequires in postgresql.spec, and depend on the build options wanted, like ldap or python activation for example. You might not need that many things for your own build though.

The RPM build is also dependent on some files obtained from some external sources, of course the source tarball, but also some pdf documentation always difficult to generate if a particular environment with jade is not set up. The files obtained from external sources can be found with the tags Source$NUM with an URL in the spec file, here is for example what happens in the case of the PDF documentation.
Source12: http://www.postgresql.org/files/documentation/pdf/%{majorversion}/%{oname}-%{majorversion}-A4.pdf

When writing this post, the 9.3 beta1 build failed only because of this PDF file not accessible:
+ cp -p $HOME/git/rpm/9.3/postgresql/EL-6/postgresql-9.3-A4.pdf .
cp: cannot stat `$HOME/git/rpm/9.3/postgresql/EL-6/postgresql-9.3-A4.pdf': No such file or directory

Note that you can as well hack a bit the spec to remove this file when generating your RPM, this needs only the removal of 2 lines.

But it was working correctly for 9.2. Note that if you want to even go deeper in the hacking of the spec file, as you might need to adjust the build process to your own environment, be sure to always use a command of that type that will generate the RPMs for you:
rpmbuild --define "_sourcedir $HOME/rpm/9.2/postgresql/EL-6" \
--define "_specdir $HOME/rpm/9.2/postgresql/EL-6" \
--define "_builddir $HOME/rpm/9.2/postgresql/EL-6" \
--define "_srcrpmdir $HOME/rpm/9.2/postgresql/EL-6" \
--define "_rpmdir $HOME/rpm/9.2/postgresql/EL-6" \
--define "dist .rhel6" -bb "postgresql-9.2.spec"

This is only a command defined in common/Makefile.global, but I think it is good to know that it is the central piece of the build process commanded by the spec file.

Once done, the following RPMs (here for 9.2) will be generated:

  • postgresql92-*.rpm, containing some client binaries (pg_dump, createdb…)
  • postgresql92-libs-*.rpm, containing some client libraries (expg, libpq, libpgtype)
  • postgresql92-server-*.rpm, containing server side binaries (initdb, postgres…)
  • postgresql92-docs-*.rpm, with the documentation
  • postgresql92-contrib-*.rpm, with all the contrib modules
  • postgresql92-devel-*.rpm, with all the development libs and headers
  • postgresql92-plperl-*.rpm, containing the extension plperl
  • postgresql92-plpython-*.rpm, containing the extension plpython
  • postgresql92-pltcl-*.rpm, containing extension pltcl
  • postgresql92-test-*.rpm, containing all the regression tests
  • postgresql92-debuginfo-*.rpm, heavy package with all the information for debugging purposes

And all you RPMs are here! Be sure to check that their content fits your needs with a command of the type “rpm -qpl”.

Posted by Michael at 2:14 pm Tagged with: , , , , , , , , , , , ,

Control dump file name in Linux

Linux No Responses »
Jan 102012

Modifying the format name of dump file in a Linux system can be made with sysctl like this.
sysctl -w kernel.core_pattern=core.%e.%p
However, making this modification command-based will not make it effective at next reboot.

In order to make the modification permanent, you need to edit the file /etc/sysctl.conf. Here the core file has the executable name %e and the process ID %p.
kernel.core_pattern = core.%e.%p

Here is a list of the possible keywords usable:

  • %p, PID of dumped process
  • %u, (numeric) real UID of dumped process
  • %g, (numeric) real GID of dumped process
  • %s, number of signal causing dump
  • %t time of dump, expressed as seconds since the Epoch, 1970-01-01 00:00:00 +0000 (UTC)
  • %h, hostname (same as nodename returned by uname(2))
  • %e, executable filename (without path prefix)
  • %c, core file size soft resource limit of crashing process (since Linux 2.6.24)
Posted by Michael at 11:31 am Tagged with: , , , , , , , , ,

RPM building

Linux No Responses »
Sep 052011

Here is a memo about RPM building in Linux environment.
First is necessary an RPM-based distribution. The most famous are Fedora and CentOS.

Let’s suppose you want to build your RPMs in the folder $RPMREPO.
You need first the correct folder tree for package creation.
mkdir $RPMREPO
cp -r /usr/src/redhat/* $RPMREPO

Then what is necessary is a spec file (Ex: spec_file.spec). It contains all the directives necessary to create the package.
You may find examples of spec files in RedHat SVN repositories like the one of PostgreSQL 9.0 package.
With a spec file, you may need a PAM file (Ex: file.pam), containing data like:
#%PAM-1.0
auth include password-auth
account include password-auth

Don’t forget that you also need a tarball (Ex: foo.tar.gz) or something equivalent containing the code.
Then copy the PAM file and the tarball inside $RPMREPO/SOURCES.
cp file.pam foo.tar.gz $RPMREPO/SOURCES
Copy the spec file to the correct folder.
cp spec_file.spec $RPMREPO/SPECS

Before building an RPM, it is necessary to set a file called .rpmmacros located in $HOME directory.
echo "%_topdir $RPMREPO" > $HOME/.rpmmacros

Then you can create the RPM with this command, assuming spec, pam and tarball files are correct.
rpmbuild -ba spec_file.spec

If no error occurred, SRPM file is located in $RPMREPO/SRPMS, RPM packages are located in $RPMREPO/RPMS/x86_64.

Here are some additional useful commands.
You can also build a RPM package from a SRPM file.
rpmbuild --rebuild $SRPM_FILE

Check content of an RPM file.
rpm -qpl $RPM_FILE

Export files of an RPM package.
rpm2cpio $RPM_FILE | cpio -idv

Posted by Michael at 12:47 pm Tagged with: , , , , , , , , ,

Partition-based install of Linux distributions

Linux No Responses »
Mar 052011

Woah, the latest version of your favorite Linux-based Operating System is available! This means that after installing it you will have at your disposition all the new fashioned applications and cool additional functionalities.

For beginners, the idea is to cut your hard disk into several pieces allowing each part to be erased or to be modified without touching anything on other parts. It permits to protect some parts of your hard disk.

However, you have to flush the entire disk of your machine just to install the new one. And it is a pain to put a backup of your 100 Go collection of music and movies on an external disk.
You spent also so much time setting your environment that doing it once again looks like a real waste of time. Everything you did to achieve an environment close to perfection will be wasted with your disk data…
Perhaps you feel that installing the latest OS is a pain just for that.

To avoid such a situation, you have the possibility to install your OS while preserving your personal data. How? Read the following lines.

Following screenshots are taken from Ubuntu 10.04 LTS installer. But it is possible to do similar operation with other distributions as well (Centos, Suze, Fedora…).
The machine used is a Virtual machine of 8 Go of Disk and 512 Mo of shared memory.

After initializing the installation with language choice, keyboard setting, or whatever, you will be asked to prepare your disk space.
For your case, choose the option Specify partitions manually.

You will be redirected to a new screen. This is the place when you will set your partitions. Now Create a new partition table on the disk you want.
In the case of my virtual machine, I have only one disk. It is also the case for most of laptops used.

You will see free space created from your hard disk (here 8 Go). This space is going to be cut into smaller pieces. So click on Add.

The first mounting point (the place in the directory folder where you base the access of this part of the disk), has to be what is called root, the lowest level. This is identified by “/”. Here we use 4 Go as disk space, feel free to enter a value adapted to your needs.
In the option “use as”, you can find a list of system files. For instance Windows uses NTFS partitions for its install. In our case, ext3 can make the deal. You can also use ext4 if you wish.
This partition is going to contain the programs you install and Operating System data. That’s why it is the primary partition.

Set up your partition as on the screen below. Then click on OK.

You will be back to the main screen, add a new partition with Add

The second partition is called swap. You have basically to put a memory space equal to 2 times the amount of shared memory of your system.
In our case, the virtual machine has 500 Mo, so we set 1000 Go. Check how much shared memory you have before setting it. After setting it click on OK.

You will be back to the main screen, so let’s go to set up the 3rd partition. Add a new partition.

This partition has to use “/home/” as a mounting point. In Linux based systems, home is the directory containing all the users data. So basically it is the partition of the disk that has to be kept independant and not erased when you reinstall a new OS.
For our case, the disk was empty, so a new partition existed. But if you reinstall a new OS on a disk already partition, don’t forget to associate “/home/” as a mounting point for this partition. This permits to keep user’s data safe with your environment settings also if you are reinstalling the same type of distribution as before (Always try to keep a gnome or KDE desktop!).
Use the rest of amount of memory for this partition.

OK, you are done, congratulations!

What you have to remember about the partitions:

  • One used as a root, mounted at “/”. This can be erased each time you want to make a fresh install.
  • Swap area, always two times equal to the shared memory of your machine. This can also be erased when you do a fresh install.
  • One partition mounted at “/home”. When you reinstall an OS, do not erase the data on it!
Posted by Michael at 2:32 pm Tagged with: , , , , , , , , , , , , ,
©2010-2013 Michael Paquier All content is ©Copyright of Otacoo.com 2010-2013. Privacy Policy - Terms of Use