Upload manuscript issue

I have the app running in production, but once the user wants to upload a manuscript, the server container in docker exits and I get the following error in console:


Here is part of my .env file for setting up a user and a bucket on my file server

S3_ACCESS_KEY_ID=nonRootUser
S3_SECRET_ACCESS_KEY=nonRootPassword
S3_BUCKET=uploads
S3_PROTOCOL=http
S3_HOST=localhost
S3_PORT=9000

Here is part of the server log (I know it is related to S3 bucket set up but I couldn’t fix it using AWS S3):
error: File Storage Healthcheck: Communication to remote file service unsuccessful UNHANDLED REJECTION at: Promise { <rejected> UnknownEndpoint: Inaccessible host: localhost’ at port 9000'. This service may not be available in the us-east-1’ region.
at Request.ENOTFOUND_ERROR (/home/node/app/node_modules/aws-sdk/lib/event_listeners.js:529:46)
at Request.callListeners (/home/node/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/home/node/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/home/node/app/node_modules/aws-sdk/lib/request.js:686:14)
at error (/home/node/app/node_modules/aws-sdk/lib/event_listeners.js:361:22)
at ClientRequest. (/home/node/app/node_modules/aws-sdk/lib/http/node.js:99:9)
at ClientRequest.emit (node:events:513:28)
at ClientRequest.emit (node:domain:489:12)
at Socket.socketErrorListener (node:_http_client:494:9)
at Socket.emit (node:events:513:28)
at Socket.emit (node:domain:489:12)
at emitErrorNT (node:internal/streams/destroy:157:8)
at emitErrorCloseNT (node:internal/streams/destroy:122:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
code: ‘UnknownEndpoint’,
region: ‘us-east-1’,
hostname: ‘localhost’,
retryable: true,
originalError: Error: connect ECONNREFUSED 127.0.0.1:9000
at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1278:16) {
errno: -111,
code: ‘NetworkingError’,
syscall: ‘connect’,
address: ‘127.0.0.1’,
port: 9000,
region: ‘us-east-1’,
hostname: ‘localhost’,
retryable: true,
time: 2024-02-18T14:36:51.279Z
},
time: 2024-02-18T14:36:51.279Z
}
} reason: UnknownEndpoint: Inaccessible host: localhost' at port 9000’. This service may not be available in the us-east-1' region.

I think you don’t have any file-hosting running.

The docker-compose.yml for development provides all the containers you need for basic functionality:

  • server
  • client (needed only in development)
  • postgres database
  • XSweet (docx to html converter)
  • DB used by XSweet
  • file-hosting (MinIO)
  • “createbucket” which runs briefly to initialise the file-hosting
  • PagedJS (PDF generator)
  • DB used by PagedJS
  • Flax (publishing CMS)
  • Anystyle (citation formatter)
  • DB used by Anystyle

The docker-compose.production.yml for production doesn’t provide most of these: it gives you:

  • server
  • XSweet
  • DB used by XSweet
  • Flax

The other containers are left out in production for either of two reasons:

  1. To prevent loss of data when you run docker-compose down: this would erase all data from the DB and file-hosting if these were included in the same docker-compose file.
  2. To allow sharing of services by multiple instances (this is more arbitrary and we could reconsider some of these exclusions).

In production, unless you have specifically created your own MinIO filehosting (or set up S3 filehosting), you probably don’t have any such container running. This would result in the error you saw. You can check what containers are running using docker ps.

You should be able to get the missing services running in separate docker-compose files. One would be for data, and this docker-compose file should be kept in a separate folder for safety: you could call it docker-compose.prod-data.yml. It would contain:

version: '3'

services:
  db:
    # ... copy this section from docker-compose.yml
    # Only if need this! It seems you already have a db running.

  fileHosting:
    # ... copy this section from docker-compose.yml

  createbucket:
    # ... copy this section from docker-compose.yml

volumes:
  minio_storage:

You would also have another docker-compose file for other services. This can safely reside in the same folder as docker-compose.production.yml, though it is probably more convenient to keep it elsewhere. It could be called docker-compose.prod-services.yml and would contain:

version: '3'

services:
  pagedjs:
    # ... copy this section from docker-compose.yml

  db-pagedjs:
    # ... copy this section from docker-compose.yml

  anystyle:
    # ... copy this section from docker-compose.yml

  db-anystyle:
    # ... copy this section from docker-compose.yml

To start Kotahi you’d first need to run

docker-compose -f path-to-prod-data/docker-compose.prod-data.yml --env-file .env up -d

Then

docker-compose -f path-to-prod-services/docker-compose.prod-services.yml --env-file .env up -d

Then

docker-compose -f docker-compose.production.yml --env-file .env up -d

I haven’t had time to test all this yet, but I think it should work.

Note: you shouldn’t also try to run development builds on the same server, as this would attempt to create containers that conflict with your production containers.

Thank you so much for the thorough reply, I think this will fix my issue!

Hi again,
I followed the set up, now I get the following error:
reason: UnknownEndpoint: Inaccessible host: filehosting' at port undefined’. This service may not be available in the us-east-1' region. at Request.ENOTFOUND_ERROR (/home/node/app/node_modules/aws-sdk/lib/event_listeners.js:529:46) at Request.callListeners (/home/node/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20) at Request.emit (/home/node/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10) at Request.emit (/home/node/app/node_modules/aws-sdk/lib/request.js:686:14) at error (/home/node/app/node_modules/aws-sdk/lib/event_listeners.js:361:22) at ClientRequest.<anonymous> (/home/node/app/node_modules/aws-sdk/lib/http/node.js:99:9) at ClientRequest.emit (node:events:513:28) at ClientRequest.emit (node:domain:489:12) at Socket.socketErrorListener (node:_http_client:494:9) at Socket.emit (node:events:513:28) at Socket.emit (node:domain:489:12) at emitErrorNT (node:internal/streams/destroy:157:8) at emitErrorCloseNT (node:internal/streams/destroy:122:3) at processTicksAndRejections (node:internal/process/task_queues:83:21) { code: 'UnknownEndpoint', region: 'us-east-1', hostname: 'filehosting', retryable: true, originalError: Error: getaddrinfo EAI_AGAIN filehosting at GetAddrInfoReqWrap.onlookup [as oncomplete] (node:dns:109:26) { errno: -3001, code: 'NetworkingError', syscall: 'getaddrinfo', hostname: 'filehosting', region: 'us-east-1', retryable: true, time: 2024-02-20T22:06:12.499Z }, time: 2024-02-20T22:06:12.499Z } I was wondering if the containers on different path can communicate? Here is my docker-compose.production.yml: version: ‘3’

services:
server:
build:
context: .
dockerfile: ./Dockerfile-production
target: server
args:
- node_env=${NODE_ENV:-production}
- server_protocol=${SERVER_PROTOCOL}
- server_host=${SERVER_HOST}
- server_port=${SERVER_PORT}
- client_protocol=${CLIENT_PROTOCOL}
- client_host=${CLIENT_HOST}
- client_port=${CLIENT_PORT}
- public_client_protocol=${PUBLIC_CLIENT_PROTOCOL}
- public_client_host=${PUBLIC_CLIENT_HOST}
- public_client_port=${PUBLIC_CLIENT_PORT}
- use_colab_email=${USE_COLAB_EMAIL:-false}
- flax_site_url=${FLAX_SITE_URL:-https://site.kotahidev.cloud68.co/}
- USE_COLAB_BIOPHYSICS_IMPORT=${USE_COLAB_BIOPHYSICS_IMPORT:-false}

ports:
  - ${SERVER_PORT:-3000}:${SERVER_PORT:-3000}
environment:
  - NODE_ENV=${NODE_ENV:-production}
  - POSTGRES_HOST=${POSTGRES_HOST}
  - POSTGRES_PORT=${POSTGRES_PORT}
  - POSTGRES_DB=${POSTGRES_DB}
  - POSTGRES_USER=${POSTGRES_USER}
  - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
  - PUBSWEET_SECRET=${PUBSWEET_SECRET}
  - CLIENT_PROTOCOL=${CLIENT_PROTOCOL}
  - CLIENT_HOST=${CLIENT_HOST}
  - CLIENT_PORT=${CLIENT_PORT}
  - PUBLIC_CLIENT_HOST=${PUBLIC_CLIENT_HOST:-0.0.0.0}
  - SERVER_PROTOCOL=${SERVER_PROTOCOL}
  - SERVER_HOST=${SERVER_HOST}
  - SERVER_PORT=${SERVER_PORT}
  - ORCID_CLIENT_ID=${ORCID_CLIENT_ID}
  - ORCID_CLIENT_SECRET=${ORCID_CLIENT_SECRET}
  - USE_SANDBOXED_ORCID=${USE_SANDBOXED_ORCID:-false}
  - USE_COLAB_EMAIL=${USE_COLAB_EMAIL:-false}
  - KOTAHI_API_TOKENS=${KOTAHI_API_TOKENS:-}
  - JOURNAL_NAME=${JOURNAL_NAME:-}
  - JOURNAL_ABBREVIATED_NAME=${JOURNAL_ABBREVIATED_NAME:-}
  - JOURNAL_HOMEPAGE=${JOURNAL_HOMEPAGE:-}
  - GOOGLE_SPREADSHEET_CLIENT_EMAIL=${GOOGLE_SPREADSHEET_CLIENT_EMAIL:-}
  - GOOGLE_SPREADSHEET_PRIVATE_KEY=${GOOGLE_SPREADSHEET_PRIVATE_KEY:-}
  - GOOGLE_SPREADSHEET_ID=${GOOGLE_SPREADSHEET_ID:-}
  - HYPOTHESIS_API_KEY=${HYPOTHESIS_API_KEY:-}
  - HYPOTHESIS_GROUP=${HYPOTHESIS_GROUP:-}
  - HYPOTHESIS_ALLOW_TAGGING=${HYPOTHESIS_ALLOW_TAGGING}
  - HYPOTHESIS_REVERSE_FIELD_ORDER=${HYPOTHESIS_REVERSE_FIELD_ORDER}
  - CROSSREF_LOGIN=${CROSSREF_LOGIN:-}
  - CROSSREF_PASSWORD=${CROSSREF_PASSWORD:-}
  - CROSSREF_REGISTRANT=${CROSSREF_REGISTRANT:-}
  - CROSSREF_DEPOSITOR_NAME=${CROSSREF_DEPOSITOR_NAME:-}
  - CROSSREF_DEPOSITOR_EMAIL=${CROSSREF_DEPOSITOR_EMAIL:-}
  - CROSSREF_PUBLICATION_TYPE=${CROSSREF_PUBLICATION_TYPE:-article}
  - CROSSREF_USE_SANDBOX=${CROSSREF_USE_SANDBOX:-false}
  - DOI_PREFIX=${DOI_PREFIX:-}
  - PUBLISHED_ARTICLE_LOCATION_PREFIX=${PUBLISHED_ARTICLE_LOCATION_PREFIX:-}
  - PUBLICATION_LICENSE_URL=${PUBLICATION_LICENSE_URL:-}
  - MANUSCRIPTS_TABLE_COLUMNS=${MANUSCRIPTS_TABLE_COLUMNS}
  - DISPLAY_SHORTID_AS_IDENTIFIER=${DISPLAY_SHORTID_AS_IDENTIFIER:-false}
  - GMAIL_NOTIFICATION_EMAIL_AUTH=${GMAIL_NOTIFICATION_EMAIL_AUTH:-}
  - GMAIL_NOTIFICATION_EMAIL_SENDER=${GMAIL_NOTIFICATION_EMAIL_SENDER:-}
  - GMAIL_NOTIFICATION_PASSWORD=${GMAIL_NOTIFICATION_PASSWORD:-}
  - PUBLISHING_WEBHOOK_URL=${PUBLISHING_WEBHOOK_URL:-}
  - PUBLISHING_WEBHOOK_TOKEN=${PUBLISHING_WEBHOOK_TOKEN:-}
  - PUBLISHING_WEBHOOK_REF=${PUBLISHING_WEBHOOK_REF:-}
  - REVIEW_SHARED=${REVIEW_SHARED:-false}
  - REVIEW_HIDE=${REVIEW_HIDE:-false}
  - NOTIFICATION_EMAIL_AUTOMATED=${NOTIFICATION_EMAIL_AUTOMATED:-false}
  - NOTIFICATION_EMAIL_CC_ENABLED=${NOTIFICATION_EMAIL_CC_ENABLED:-false}
  - SERVICE_PAGEDJS_CLIENT_ID=${SERVICE_PAGEDJS_CLIENT_ID}
  - SERVICE_PAGEDJS_SECRET=${SERVICE_PAGEDJS_SECRET}
  - SERVICE_PAGEDJS_PROTOCOL=${SERVICE_PAGEDJS_PROTOCOL}
  - SERVICE_PAGEDJS_HOST=${SERVICE_PAGEDJS_HOST}
  - SERVICE_PAGEDJS_PORT=${SERVICE_PAGEDJS_PORT}
  - SERVICE_ANYSTYLE_CLIENT_ID=${SERVICE_ANYSTYLE_CLIENT_ID}
  - SERVICE_ANYSTYLE_SECRET=${SERVICE_ANYSTYLE_SECRET}
  - SERVICE_ANYSTYLE_PROTOCOL=${SERVICE_ANYSTYLE_PROTOCOL}
  - SERVICE_ANYSTYLE_HOST=${SERVICE_ANYSTYLE_HOST}
  - SERVICE_ANYSTYLE_PORT=${SERVICE_ANYSTYLE_PORT}
  - S3_PROTOCOL=${S3_PROTOCOL:-http}
  - S3_HOST=fileHosting
  - S3_PORT=${S3_PORT:-}
  - S3_ACCESS_KEY_ID=${S3_ACCESS_KEY_ID:-nonRootUser}
  - S3_SECRET_ACCESS_KEY=${S3_SECRET_ACCESS_KEY:-nonRootPassword}
  - S3_BUCKET=${S3_BUCKET:-uploads}
  - MINIO_ROOT_USER=${MINIO_ROOT_USER:-admin}
  - MINIO_ROOT_PASSWORD=${MINIO_ROOT_PASSWORD:-superSecretAdminPassword}
  - MINIO_CONSOLE_PORT=${MINIO_CONSOLE_PORT:-9001}
  - AUTO_IMPORT_HOUR_UTC=${AUTO_IMPORT_HOUR_UTC:-}
  - DISABLE_EVENT_NOTIFICATIONS=${DISABLE_EVENT_NOTIFICATIONS:-}
  - ARCHIVE_PERIOD_DAYS=${ARCHIVE_PERIOD_DAYS:-}
  - ALLOW_MANUAL_IMPORT=${ALLOW_MANUAL_IMPORT:-}
  - SEMANTIC_SCHOLAR_IMPORTS_RECENCY_PERIOD_DAYS=${SEMANTIC_SCHOLAR_IMPORTS_RECENCY_PERIOD_DAYS:-}
  - TEAM_TIMEZONE=${TEAM_TIMEZONE}
  - SERVICE_XSWEET_CLIENT_ID=59a3392b-0c4f-4318-bbe2-f86eff6d3de4
  - SERVICE_XSWEET_SECRET=asldkjLKJLaslkdf897kjhKUJH
  - SERVICE_XSWEET_PROTOCOL=${SERVICE_XSWEET_PROTOCOL:-http}
  - SERVICE_XSWEET_HOST=${SERVICE_XSWEET_HOST:-xsweet}
  - SERVICE_XSWEET_PORT=${SERVICE_XSWEET_PORT:-3004}
  - FLAX_EXPRESS_PORT=${FLAX_EXPRESS_PORT}
  - FLAX_EXPRESS_HOST=${FLAX_EXPRESS_HOST:-kotahi-flax-site}
  - FLAX_EXPRESS_PROTOCOL=${FLAX_EXPRESS_PROTOCOL:-http}
  - FLAX_CLIENT_ID=${FLAX_CLIENT_ID}
  - FLAX_CLIENT_SECRET=${FLAX_CLIENT_SECRET}
  - FLAX_CLIENT_API_URL=${FLAX_CLIENT_API_URL}
  - FLAX_SITE_URL=${FLAX_SITE_URL}
  - USE_APERTURE_EMAIL=${USE_APERTURE_EMAIL}
  - INSTANCE_GROUPS=${INSTANCE_GROUPS:-kotahi:journal}
  - USE_COLAB_BIOPHYSICS_IMPORT=${USE_COLAB_BIOPHYSICS_IMPORT:-false}

volumes:
  - ./uploads:/home/node/app/uploads
  - ./config:/home/node/app/config

xsweet:
depends_on:
- db-xsweet
image: cokoapps/xsweet:2.2.1
ports:
- ${SERVICE_XSWEET_PORT:-3004}:${SERVICE_XSWEET_PORT:-3004}
environment:
- PUBSWEET_SECRET=xsweet_dev_secret
- POSTGRES_USER=xsweet_user_dev
- POSTGRES_PASSWORD=xsweet_user_password
- POSTGRES_HOST=db-xsweet
- POSTGRES_DB=xsweet_dev
- POSTGRES_PORT=5432
- SERVER_PORT=${SERVICE_XSWEET_PORT:-3004}
- NODE_ENV=development
entrypoint:
[
‘sh’,
‘scripts/wait-for-it’,
‘db-xsweet:5432’,
‘–’,
‘sh’,
‘scripts/setupDevServer.sh’,
]
command: [‘node’, ‘server/startServer.js’]

db-xsweet:
image: postgres:14.4-alpine
environment:
- POSTGRES_USER=xsweet_user_dev
- POSTGRES_DB=xsweet_dev
- POSTGRES_PASSWORD=xsweet_user_password
volumes:
- ./scripts/init-pgboss.sql:/docker-entrypoint-initdb.d/init-pgboss.sql

db-server:
image: postgres:14.4-alpine
ports:
- 15432:5432
volumes:
- server_db_data:/var/lib/postgresql/data
environment:
- POSTGRES_PASSWORD=kotahidev
- POSTGRES_USER=kotahidev
- POSTGRES_DB=kotahidev

kotahi-flax-site:
image: cokoapps/kotahi-flax:0.2.33
depends_on:
- server
ports:
- ${FLAX_EXPRESS_PORT:-8082}:${FLAX_EXPRESS_PORT:-3000}
- ${FLAX_SITE_PORT:-8081}:80

environment:
  - FLAX_EXPRESS_PORT=${FLAX_EXPRESS_PORT:-3000}
  - FLAX_CLIENT_ID=${FLAX_CLIENT_ID}
  - FLAX_CLIENT_SECRET=${FLAX_CLIENT_SECRET}
  - FLAX_CLIENT_API_URL=${FLAX_CLIENT_API_URL}

volumes:
server_db_data:
`

Putting all of them next to each other in docker-compose.production.yml fixed the issue!

1 Like