Installing psycopg2-binary with Python:3.6.4-alpine doesn't work

[barrachri]

When you try to install the binary version of the lib you get the source version error.

$ docker run -it --rm python:3.6.4-alpine sh
$ pip install psycopg2-binary

Collecting psycopg2-binary
  Downloading psycopg2-binary-2.7.4.tar.gz (426kB)
    100% |████████████████████████████████| 430kB 52kB/s
    Complete output from command python setup.py egg_info:
    running egg_info
    creating pip-egg-info/psycopg2_binary.egg-info
    writing pip-egg-info/psycopg2_binary.egg-info/PKG-INFO
    writing dependency_links to pip-egg-info/psycopg2_binary.egg-info/dependency_links.txt
    writing top-level names to pip-egg-info/psycopg2_binary.egg-info/top_level.txt
    writing manifest file 'pip-egg-info/psycopg2_binary.egg-info/SOURCES.txt'

    Error: pg_config executable not found.

    pg_config is required to build psycopg2 from source.  Please add the directory
    containing pg_config to the $PATH or specify the full executable path with the
    option:

        python setup.py build_ext --pg-config /path/to/pg_config build ...

    or with the pg_config option in 'setup.cfg'.

    If you prefer to avoid building psycopg2 from source, please install the PyPI
    'psycopg2-binary' package instead.

    For further information please check the 'doc/src/install.rst' file (also at
    <http://initd.org/psycopg/docs/install.html>).

[dvarrazzo]

Alpine Linux use musl libc and manylinux wheels don't support it: see e.g. pypa/manylinux#37. Not a psycopg problem but a lack of a standard for it, such as PEP 513 defininig manylinux1. Happy to support Alpine as soon as there will be a standard and tools for it.

[nourspace]

I found this solution which worked quite fine.

FROM python:3.6-alpine

RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev

pip install psycopg2-binary

---

Edit

The above compiles the package from source, so I would rather

FROM python:3.6-alpine
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev
pip install psycopg2

not to get confused.

[barrachri]

But I think you are pseudo-using the binary package and building from the source at the end.

[dvarrazzo]

@nourchawich your "solution" is about compiling psycopg from source, which is totally supported. Binary packages are not (for the reason described above).

[nourspace]

Thank for the clarification @barrachri and @dvarrazzo

I just noticed what was actually happening. All I wanted was to have psycopg2 (regardless whether binary or not) installed on an alpine based image, hence considering it a solution.

[cscetbon]

@nourchawich after it's been built, I suppose we can get rid of those dev packages ?

[siaarzh]

@nourchawich after it's been built, I suppose we can get rid of those dev packages ?

Yes, but make sure the apt add and apt del commands are in one layer. Otherwise you won't make the image any smaller.
E.g.:

RUN apk add --no-cache --virtual .build-deps \
    gcc \
    python3-dev \
    musl-dev \
    postgresql-dev \
    && pip install --no-cache-dir psycopg2 \
    && apk del --no-cache .build-deps

[cscetbon]

Yeah I did, thanks !

[Cerebus]

Since this is the top google hit for this problem, use a multi-stage build to minimize the final image size.

FROM python:3.6-alpine as base                                                                                                
                                                                                                                              
FROM base as builder                                                                                                          
                                                                                                                              
RUN mkdir /install                                                                                                            
RUN apk update && apk add postgresql-dev gcc python3-dev musl-dev                                                             
WORKDIR /install                                                                                                              
COPY requirements.txt /requirements.txt                                                                                       
RUN pip install --install-option="--prefix=/install" -r /requirements.txt                                                     
                                                                                                                              
FROM base                                                                                                                     
                                                                                                                              
COPY --from=builder /install /usr/local                                                                                       
COPY src /app                                                                                                         
RUN apk --no-cache add libpq                                                                                                  
WORKDIR /app

[hugobuddel]

This is still a bit unclear to me. The point of installing psycopg2-binary is that compilers like gcc etc. are not necessary right?

I'm getting the same message as the original poster. When trying to install psycopg2-binary, it complains about not finding gcc and suggests to, well, install psycopg2-binary (which is what I'm trying to do).

[hugobuddel]

I'm on Centos 7.4, and apparently I had to yum install postgresql-devel to make psycopg2-binary to work.

[Cerebus]

@hugobuddel The problem is when pip decides it can't install the psycopg2-binary wheel it falls back on building from source, which naturally requires dev packages. On alpine that's because of muslc instead of glibc. I should think this shouldn't happen on CentOS but maybe there's another conflict.