11.1 - 2018-11-08(木) 09:00:00
11.1 is the latest release in the 11 series.
カテゴリー: postgresql

10.6 - 2018-11-08(木) 09:00:00
10.6 is the latest release in the 10 series.
カテゴリー: postgresql

9.6.11 - 2018-11-08(木) 09:00:00
9.6.11 is the latest release in the 9.6 series.
カテゴリー: postgresql

9.5.15 - 2018-11-08(木) 09:00:00
9.5.15 is the latest release in the 9.5 series.
カテゴリー: postgresql

9.4.20 - 2018-11-08(木) 09:00:00
9.4.20 is the latest release in the 9.4 series.
カテゴリー: postgresql

9.3.25 - 2018-11-08(木) 09:00:00
9.3.25 is the latest release in the 9.3 series. This version is unsupported!
カテゴリー: postgresql

Hernan Resnizky: Machine Learning in PostgreSQL Part 1: Kmeans clustering

planet postgresql - 2018-11-07(水) 18:00:03
Machine Learning in 10 Lines

Every person that reads newspapers, magazines or any other media of general interest has at least a basic idea of what Machine Learning is. And this is not only a fashion, Machine Learning is already part of our everyday life and will be much more in the future: from personalized advertisement on the Internet to robot dentists or autonomous cars, Machine Learning seems to be some kind of super power capable of everything.


But, what is Machine Learning really? It is mainly a set of statistical algorithms that, based on existing data, are capable of deriving insights out of them. These algorithms are basically divided into two families, supervised and unsupervised learning. In supervised learning, the objective is to perform some kind of prediction, such as, for example, if an e-mail message is spam or not (classification), how many beers will be sold next week in a supermarket (regression), etc. Unsupervised Learning, on the contrary, focuses on answering the question how are my cases divided in groups? What these algorithms do (each of them with their particularities) is to bring similar items as close as possible and keep items that differ  from each other as far as possible.

The popularisation of Machine Learning revolutionized the way we do business. Regardless if you are talking of a 10 or 10,000 employees company, if you do not make use of your data to make decisions, you are definitely running behind your competitors.


Machine Learning without leaving the Database

Relational Databases are definitely the most essential tools when it comes to data persistence. Although there are other alternatives which could be suitable for certain purposes, there is probably no company with at least a minimal IT Infrastructure that doesn’t have a database.

So if every company has a database, it contains data that is worth using. This means that every company has the opportunity to improve its decision-making process with minimal effort through the use of machine learning. However, he drawbac

カテゴリー: postgresql

William Ivanski: OmniDB debugger for PostgreSQL 11

planet postgresql - 2018-11-06(火) 22:58:04

PostgreSQL 11 was released recently, with exciting new features. One of them is the ability to write SQL procedures that can perform full transaction management, enabling developers to create more advanced server-side applications. SQL procedures can be created using the CREATE PROCEDURE command and executed using the CALL command. Since OmniDB 2.3.0 it is possible to debug PostgreSQL PL/pgSQL functions. Support to PostgreSQL 11 functions and procedures was added in OmniDB 2.11.0.

Last week we released OmniDB 2.12.0 with nice new features and a new revamped visual, so I’m going to show you how OmniDB 2.12.0 can debug PostgreSQL 11 procedures.

First of all, if you have not done that already, download and install a binary PostgreSQL library called omnidb_plugin and enable it in PostgreSQL’s config file. The debugger also uses a special schema with special tables to control the whole debugging process. This can be manually created or with an extension. For more details on the installation, please refer to the instructions. You can also refer to the documentation about the debugger.

Creating some tables in OmniDB

For our tests, let’s create 2 simple tables, foo and bar. Let’s do that using the OmniDB Console Tab:


Creating a procedure with transaction management

Note that OmniDB has a Procedures node in the tree view. Right-click on it, then click on Create Procedure. It will open a Query Tab with a SQL template showing basic SQL syntax to create a procedure.

If you want to know more about procedures, you can read online documentation without leaving OmniDB. Simple click on Procedures -> Doc: Procedures and a browser tab will be open for you already pointing to the documentation page:

Now let’s go back to the Create Procedure tab and change the code to actually create a procedure, like this:

CREATE OR REPLACE PROCEDURE public.prc_test ( p INTEGER ) LANGUAGE plpgsql AS $procedure$ BEGIN F[...]
カテゴリー: postgresql

Magnus Hagander: Tracking foreign keys throughout a schema

planet postgresql - 2018-11-05(月) 22:44:41

I recently ran into the need with a customer to track the usage of a specific key throughout the schema. Basically, "what are all the tables and columns referencing this key, directly or indirectly". Luckily, with a little bit of catalog query, that's not hard:

WITH RECURSIVE what (tbl) AS ( VALUES ('') ), t (oid, key, constrid) AS ( SELECT tbl::regclass::oid, conkey, NULL::oid FROM what INNER JOIN pg_constraint ON (contype='p' AND conrelid=tbl::regclass) UNION ALL SELECT conrelid, conkey, c.oid FROM pg_constraint c INNER JOIN t ON (c.confrelid=t.oid AND c.confkey=t.key) WHERE contype='f' ) SELECT nspname, relname, key, ARRAY( SELECT attname FROM pg_attribute a WHERE a.attrelid=t.oid AND attnum=ANY(key) ) FROM t INNER JOIN pg_class cl ON cl.oid=t.oid INNER JOIN pg_namespace n ON n.oid=cl.relnamespace

The output can be similar to:

nspname | relname | key | array ---------+---------+-----+------- public | tt | {1} | {ttt} public | foo1 | {1} | {a} public | foo2 | {3} | {z}

for a single column key (tt being the table with the primary key in, and the foo1 and foo2 tables referencing it directly or through the other one), or:

nspname | relname | key | array ---------+---------+-------+------- public | m1 | {1,2} | {a,b} public | m2 | {1,2} | {a,b}

for a multi-column foreign key.

In this particular use-case, it was an efficient way to track down key usage where naming standards for using the key had not always been followed. And of course, we also found a couple of cases where the column had the correct name but lacked the actual FOREIGN KEY definition, but that was done by just looking at the column names.

カテゴリー: postgresql

Bruce Momjian: Submitting Talks to Conferences

planet postgresql - 2018-11-05(月) 19:00:02

Having attended many conferences, I have a few suggestions on how to submit successful conference talks. First, determine the type of conference. Then, try to submit talks that match the conference type; possible topics include:

  • New Postgres features
  • User cast studies
  • Internals
  • New workloads
  • Performance
  • Application development

Of course, only some of these topics match specific types of conferences.

Second, submit multiple talks. It is very possible that someone better known than you, or someone with a better abstract, will also submit to the conference. By submitting more than one topic, you increase your chances of submitting something unique and interesting.

Continue Reading »

カテゴリー: postgresql

Laurenz Albe: Killed index tuples

planet postgresql - 2018-11-05(月) 18:00:16
© Laurenz Albe 2018


Since I only recently learned about the concept of “killed index tuples”, I thought there might be some others who are not yet familiar with this interesting PostgreSQL concept.

This may give you an explanation the next time you encounter wildly varying execution times for the same execution plan of the same PostgreSQL query.

Before we look more closely at the index, let’s review the life cycle of a table row version (“heap tuple”).

Life, death and visibility in the table heap

It is widely known that the visibility of heap tuples is determined by the system columns xmin and xmax (though there is more to xmax than meets the eye). A heap tuple is “dead” if its xmax is less than the xmin of all active transactions.

Now xmin and xmax are only valid if the respective transactions have been marked committed in the “commit log”. Consequently, any transaction that needs to know if it can see a tuple has to consult the commit log. To save future readers that extra work, the first one that consults the commit log will save the information in the tuple’s “hint bits”.

Dead tuples are eventually reclaimed by VACUUM.

This is all fairly well known, but how is the situation with index entries?

Life, death and visibility in the index

To avoid redundancy and to keep index tuples small, the visibility information is not stored in the index.
The status of an index tuple is determined by the heap tuple it points to, and both are removed by VACUUM at the same time.

As a consequence, an index scan has to inspect the heap tuple to determine if it can “see” an entry. This is the case even if all the columns needed are in the index tuple itself. Even worse, this “heap access” will result in random I/O, which is not very efficient on spinning disks.

This makes index scans in PostgreSQL more expensive than in other database management systems that use a different architecture. To mitigate that, several features have been introduced over the years:

  • PostgreSQL 8.1 introduced the “bitmp index scan”. This scan method fi
カテゴリー: postgresql

Hubert 'depesz' Lubaczewski: Foreign Key to partitioned table – part 3

planet postgresql - 2018-11-05(月) 03:33:28
Previously I tested performance of pl/PgSQL coded foreign keys to partitioned table. Now, let's see if I can make creation of them a bit easier. Using the same schema as before, I see that adding actual fkeys is pretty complicated. I need to create two separate functions, and four triggers, remembering what goes where. This … Continue reading "Foreign Key to partitioned table – part 3"
カテゴリー: postgresql

Andreas Scherbaum: Using Makefiles to build PostgreSQL

planet postgresql - 2018-11-04(日) 06:00:00

Andreas 'ads' Scherbaum

For a long time I was using a Makefile to quickly build, start, stop and then wipe a predefined PostgreSQL version. That comes handy if you just want to test something on an older version, without actually installing the software. Everything happens in a single directory, even a different port is assigned.

When I needed that setup recently, I ran into unrelated build errors:

relpath.c:21:10: fatal error: catalog/pg_tablespace_d.h: No such file or directory #include "catalog/pg_tablespace_d.h" ^~~~~~~~~~~~~~~~~~~~~~~~~~~ compilation terminated.

Can't be - pg_tablespace_d.h is included in the tarball I'm using.



Continue reading "Using Makefiles to build PostgreSQL"
カテゴリー: postgresql

Abdul Yadi: pgAdmin3 Adjustment for PostgreSQL 11.0

planet postgresql - 2018-11-03(土) 15:01:57

What is my favourite PostgreSQL GUI-admin tool? pgAdmin3. I love its light weight user interface and simple navigation. Thanks to BigSQL Development Team for surviving the tool from freeze.

With PostgreSQL release 11.0, here is my patch file corresponding catalog table changes: pgadmin3-patch-text-file

First, clone pgAdmin3 project: clone git clone

Then, apply the patch: patch -p0 -i [patch-text-file]

Oldies but goldies.

カテゴリー: postgresql

Rafia Sabih: My experience at PGConf Europe 2018

planet postgresql - 2018-11-02(金) 13:58:00
It was my first time at PGConf Europe this year, like many other firsts it was special, hence the blog.
Let's start with some of the basics, PostgreSQL conferences are held in a somewhat regional basis. There are many of them like,  PGConf India, PGConf USA, PGConf Europe, PGConf Asia, and then there are other one day events called PgDays. Coming back to PGConf Europe 2018,  it was organised from 23-26 October in Lisbon Marriott, Lisbon.
My talk 'Parallel Query in PG: how not to (mis)use it?' was scheduled on the first slot of last day. So, I had enough time to analyse and study the audience and prepare accordingly. But, first things first...
The conference started with a one day training session on 22 Oct, one has to buy different tickets for training and conference. You get a free registration for the conference only if you're the speaker. I wasn't part of the training session, hence will not be discussing anything about it. This was my day to rest and try the Portugal cuisine.
The next day was the start of the conference. It was opened by Magnus Hagander covering the logistics and introducing us to the conference halls, etc., must say it was one entertaining start. The next was the keynote by Paul Ramsey. The keynote was my first comprehensive introduction to PostGIS. Further, there was a nice snack buffet arranged in the lobby, and this was my time to know more people, the most exciting part of any conference. I happened to catch Tom Lane!
Henceforth, I was forced to take some difficult decisions like which talk to attend, since there were three parallel sessions going on. There was such a variety of areas covered in the conference and most of them have amazing presentations, that it made me greedy and hate the idea of parallel sessions.
To keep the discussion short, I enjoyed being exposed to some of the new areas and uses of postgres like, challenges of using postgres on cloud, multi-column indexes, pluggable storage, benchmarking,  efficient query planning in latest PG, new and old features of pos[...]
カテゴリー: postgresql

Bruce Momjian: Users vs. Developers

planet postgresql - 2018-11-01(木) 20:45:01

Some open source projects have a distinction between the developers of the open source software and its users. Since Postgres was originally developed in a university, and none of the university developers continued when Internet-based development started in 1996, all our active developers see themselves as stewards of code developed before we arrived. This causes a flatter organizational structure and helps to forge closer user/developer ties.

カテゴリー: postgresql

PostgreSQL10.5のドキュメント日本語版を公開 (2018-11-01) news - 2018-11-01(木) 10:29:09
PostgreSQL10.5のドキュメント日本語版を公開 (2018-11-01) kuwamura 2018/11/01 (木) - 10:29
カテゴリー: postgresql

Daniel Pocock: RHL'19 St-Cergue, Switzerland, 25-27 January 2019

planet postgresql - 2018-11-01(木) 06:06:34

(translated from original French version)

The Rencontres Hivernales du Libre (RHL) (Winter Meeting of Freedom) takes place 25-27 January 2019 at St-Cergue. invites the free software community to come and share workshops, great meals and good times.

This year, we celebrate the 5th edition with the theme «Exploit».

Please think creatively and submit proposals exploring this theme: lectures, workshops, performances and other activities are all welcome.

RHL'19 is situated directly at the base of some family-friendly ski pistes suitable for beginners and more adventurous skiers. It is also a great location for alpine walking trails.

Why, who?

RHL'19 brings together the forces of freedom in the Leman basin, Romandy, neighbouring France and further afield (there is an excellent train connection from Geneva airport). Hackers and activists come together to share a relaxing weekend and discover new things with free technology and software.

If you have a project to present (in 5 minutes, an hour or another format) or activities to share with other geeks, please send an email to or submit it through the form.

If you have any specific venue requirements please contact the team.

You can find detailed information on the event web site.

Please ask if you need help finding accommodation or any other advice planning your trip to the region.

カテゴリー: postgresql

Craig Kerstiens: Materialized views vs. Rollup tables in Postgres

planet postgresql - 2018-11-01(木) 04:12:00

Materialized views were a long awaited feature within Postgres for a number of years. They finally arrived in Postgres 9.3, though at the time were limited. In Postgres 9.3 when you refreshed materialized views it would hold a lock on the table while they were being refreshed. If your workload was extremely busines hours based this could work, but if you were powering something to end-users this was a deal breaker. In Postgres 9.4 we saw Postgres achieve the ability to refresh materialized views concurrently. With this we now have fully baked materialized view support, but even still we’ve seen they may not always be the right approach.

What is a view?

For those of you that aren’t database experts we’re going to backup a little bit. To know what a materialized view is we’re first going to look at a standard view. A view is a defined query that you can query against as if it were a table. Views are especially helpful when you have complex data models that often combine for some standard report/building block. We’ll look at an example in just a moment as we get to a materialized views.

Views are great for simplifying copy/paste of complex SQL. The downside is that each time a view is executed the results are recomputed. For large datasets this can cause scanning of a lot of data, invalidate your cache, and in general just be slow. Enter materialized views

Materializing your views

Let’s start with an example schema that could contain a lot of raw data. In this case a very basic web analytics tool that records pageview, the time it occurred, and the session id of the user.

CREATE TABLE pageviews ( id bigserial, page text, occurred_at timestamptz, session_id bigint );

There are a number of different views that could be very common based on this raw data. And if we have a real-time dashboard we’re powering it can quickly become unfeasible to query this raw data as a query would take too long. Instead we can do some rollups with materialized views:

CREATE MATERIALIZED VIEW rollups AS SELECT date_trunc('day') a[...]
カテゴリー: postgresql

Sebastian Insausti: How to Create a Single Endpoint for your PostgreSQL Replication setup using HAProxy

planet postgresql - 2018-10-31(水) 23:41:48

Managing traffic to the database can get harder and harder as it increases in amount and the database is actually distributed across multiple servers. PostgreSQL clients usually talk to a single endpoint. When a primary node fails, the database clients will keep retrying the same IP. In case you have failed over to a secondary node, the application needs to be updated with the new endpoint. This is where you would want to put a load balancer between the applications and the database instances. It can direct applications to available/healthy database nodes and failover when required. Another benefit would be to increase read performance by using replicas effectively. It is possible to create a read-only port that balances reads across replicas. In this blog, we will cover HAProxy. We’ll see what is, how it works and how to deploy it for PostgreSQL.

What is HAProxy?

HAProxy is an open source proxy that can be used to implement high availability, load balancing and proxying for TCP and HTTP based applications.

As a load balancer, HAProxy distributes traffic from one origin to one or more destinations and can define specific rules and/or protocols for this task. If any of the destinations stops responding, it is marked as offline, and the traffic is sent to the rest of the available destinations.

How to install and configure HAProxy manually

To install HAProxy on Linux you can use the following commands:

On Ubuntu/Debian OS:

$ apt-get install haproxy -y

On CentOS/RedHat OS:

$ yum install haproxy -y

And then we need to edit the following configuration file to manage our HAProxy configuration:

$ /etc/haproxy/haproxy.cfg

Configuring our HAProxy is not complicated, but we need to know what we are doing. We have several parameters to configure, depending on how we want HAProxy to work. For more information, we can follow the documentation about the HAProxy configuration.

Let's look at a basic configuration example. Suppose that you have the following database topology:

Database Topology Example

We want to create a HA

カテゴリー: postgresql