Ни один модуль с именем 'psycopg2' + ошибка при установке

Я разработал питоновый скрипт, который импортирует данные в мою Postgres DB, и он прекрасно работает на моей локальной машине, но когда я передал его на сервер, он дает сбой с:

Traceback (most recent call last):
  File "./importer.py", line 1, in <module>
    import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

вот питоновый скрипт:

import psycopg2
import glob, os
import zipfile
import ntpath

## db connection string and objects
t_host = "localhost"
t_dbname = "*****"
t_username = "******"
t_password = "******"
t_port = 5432
t_schema = "******"
t_stageTable = "******"

## working directories
inputDir = "./incoming"
outputDir = "./processed"

## functions definition
def clearDirectory(dir):
    for csvFile in glob.glob(dir+"/*.csv"):
        os.remove(csvFile)

def stripQuotes(str):
    return str.strip('"')

def uploadFile(file):
    db_conn = psycopg2.connect(host=t_host, port=t_port, dbname=t_dbname, user=t_username, password=t_password)
    db_cursor = db_conn.cursor()

    f_contents = open(file, 'r')

    for row in f_contents:
        record = row.rstrip().split(",")
        record.append('"'+ntpath.basename(file)+'"')
        insert_query = "INSERT INTO ...")

    #commit after the whole file is loaded
    db_conn.commit()    

    #close connection
    db_cursor.close()
    db_conn.close()

def processFiles(dirIn, dirOut):
    for file in glob.glob(dirIn+"/*.zip"):
        #extract files
        with zipfile.ZipFile(file, 'r') as zip_ref:
            zip_ref.extractall(dirOut)

    for file in glob.glob(dirOut+"/*.csv"):
        uploadFile(file)

    #move .zip files to processed folder
    for file in glob.glob(dirIn+"/*.zip"):
        os.replace(file, dirOut+"/"+ntpath.basename(file))


###################################
## actual processing
###################################
#clean old files first (just to be sure)
clearDirectory(outputDir)

#loop through zip files in the incoming folder
processFiles(inputDir, outputDir)

#clean again
clearDirectory(outputDir)
###################################

я пытался установить отсутствующую библиотеку psycopg2 с помощью pip3

sudo pip3 установить psycopg2

но я получаю эту странную ошибку:

Collecting psycopg2
  Using cached psycopg2-2.8.6.tar.gz (383 kB)
Building wheels for collected packages: psycopg2
  Building wheel for psycopg2 (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-uwxpzq13
       cwd: /tmp/pip-install-80qwhbn5/psycopg2/
  Complete output (40 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-x86_64-3.8
  creating build/lib.linux-x86_64-3.8/psycopg2
  copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
  copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
  running build_ext
  building 'psycopg2._psycopg' extension
  creating build/temp.linux-x86_64-3.8
  creating build/temp.linux-x86_64-3.8/psycopg
  x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.6 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120006 -DHAVE_LO64=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
  In file included from psycopg/psycopgmodule.c:28:
  ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
     36 | #include <libpq-fe.h>
        |          ^~~~~~~~~~~~
  compilation terminated.
  
  It appears you are missing some prerequisite to build the package from source.
  
  You may install a binary package by installing 'psycopg2-binary' from PyPI.
  If you want to install psycopg2 from source, please install the packages
  required for the build and try again.
  
  For further information please check the 'doc/src/install.rst' file (also at
  <https://www.psycopg.org/docs/install.html>).
  
  error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
  ----------------------------------------
  ERROR: Failed building wheel for psycopg2
  Running setup.py clean for psycopg2
Failed to build psycopg2
Installing collected packages: psycopg2
    Running setup.py install for psycopg2 ... error
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-4g11q7zi/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/psycopg2
         cwd: /tmp/pip-install-80qwhbn5/psycopg2/
    Complete output (40 lines):
    running install
    running build
    running build_py
    creating build
    creating build/lib.linux-x86_64-3.8
    creating build/lib.linux-x86_64-3.8/psycopg2
    copying lib/_json.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/extensions.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/_ipaddress.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/extras.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/errorcodes.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/compat.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/sql.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/_range.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/tz.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/_lru_cache.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/errors.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/__init__.py -> build/lib.linux-x86_64-3.8/psycopg2
    copying lib/pool.py -> build/lib.linux-x86_64-3.8/psycopg2
    running build_ext
    building 'psycopg2._psycopg' extension
    creating build/temp.linux-x86_64-3.8
    creating build/temp.linux-x86_64-3.8/psycopg
    x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -DPSYCOPG_VERSION=2.8.6 (dt dec pq3 ext lo64) -DPG_VERSION_NUM=120006 -DHAVE_LO64=1 -I/usr/include/python3.8 -I. -I/usr/include/postgresql -I/usr/include/postgresql/12/server -c psycopg/psycopgmodule.c -o build/temp.linux-x86_64-3.8/psycopg/psycopgmodule.o -Wdeclaration-after-statement
    In file included from psycopg/psycopgmodule.c:28:
    ./psycopg/psycopg.h:36:10: fatal error: libpq-fe.h: No such file or directory
       36 | #include <libpq-fe.h>
          |          ^~~~~~~~~~~~
    compilation terminated.
    
    It appears you are missing some prerequisite to build the package from source.
    
    You may install a binary package by installing 'psycopg2-binary' from PyPI.
    If you want to install psycopg2 from source, please install the packages
    required for the build and try again.
    
    For further information please check the 'doc/src/install.rst' file (also at
    <https://www.psycopg.org/docs/install.html>).
    
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"'; __file__='"'"'/tmp/pip-install-80qwhbn5/psycopg2/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-4g11q7zi/install-record.txt --single-version-externally-managed --compile --install-headers /usr/local/include/python3.8/psycopg2 Check the logs for full command output.

никогда не видел этого раньше. . есть идеи, что происходит?

Я даже пытался установить psycopg2-бинарный файл ... но все равно пи-скрипт заканчивается той же ошибкой -> нет модуля ...

EDIT: @Simon Aldrich

sudo apt install libpg-dev заканчивается:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 libpq-dev : Depends: libpq5 (= 12.6-0ubuntu0.20.04.1) but 13.2-1.pgdg20.04+1 is to be installed
E: Unable to correct problems, you have held broken packages.

когда я пытался убрать libpq5. sudo apt cleange libpq5

it says:

Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  libapache2-mod-wsgi-py3 libboost-filesystem1.71.0 libboost-thread1.71.0 libllvm10
Use 'sudo apt autoremove' to remove them.
The following packages will be REMOVED:
  libpq5* pgadmin4* pgadmin4-desktop* pgadmin4-server* pgadmin4-web* pgagent* postgresql* postgresql-12* postgresql-client-12*
  postgresql-contrib*
0 upgraded, 0 newly installed, 10 to remove and 0 not upgraded.

I don't want the pgadmin to be removed :(

0
задан 15 April 2021 в 13:04

1 ответ

Похоже, вы пропустили пакет libpq-dev, в котором будут нужные вам заголовки.

1
ответ дан 23 April 2021 в 23:22

Другие вопросы по тегам:

Похожие вопросы: