Request entity too Large Nginx server

Server
,
1

Hi RealRancor and others! :slight_smile:
Congrats on the New Forum!

Steps to reproduce
1. 32G of files in upload queue
2. External drive at 1T with 869G remaining.
3. Server sits behind a proxy redirect also Nginx.

Expected behaviour
Large files can upload.

Actual behaviour
Since O.C-8, all subsequent versions of O.C have failed to upload large files on my Raspberry Pi running Archlinux ARM. Displays an error in the client(s) as Request entity too large. And in my error log as:

01:29:47 [error] 3074#3074: *780 client intended to send too large body: 10000000 bytes, client: 10.0.1.10, server: www.example.com, request: "PUT /owncloud/remote.php/webdav/<filename>"

What is strange here is the amount of bytes is 10MB.

I have configured everything according to this guide: git. And, all read other historical info I could find pertaining to similar issues Large file issue and the PHP bug. As well as trying 2G as a limit in case of it being a Pi related issue. Now hoping someone has a fresh insight into this issue?

Server configuration
Operating system: Archlinux ARM 4.4.17-2-ARCH
Web server: Nginx 1.10.1 (FPM/FastCGI)
Database: mysql 10.1.16-MariaDB
PHP version: PHP 7.0.9
ownCloud version: 9.1.0-1
Updated from an older ownCloud or fresh install: update from 9.0
ownCloud log: last from April unrelated to this issue Mysql permission error.

Special configuration (external storage, external authentication, reverse proxy, server-side-encryption):

  • External storage, with a tmp_dir file configured to it (both in PHP and Nginx), mentioned above.
  • reverse proxy yes Nginx with client_max_body_size 16G and buffering configured as specified.

Integrity status for oC9+

No errors have been found.

Configurations redirect proxy:

location /owncloud/ {
        proxy_pass https://10.0.1.35;

        ### Set headers ###
        proxy_set_header Accept-Encoding "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;

        proxy_buffering off;
        proxy_request_buffering off;

        #add_header Front-End-Https on;

        ### Options ##
        client_max_body_size 16400M;

        ### Set timeouts ###
        proxy_read_timeout 600s;
        proxy_send_timeout 600s;
        proxy_connect_timeout 600s;

Main Config: 

  # set max upload size
  client_max_body_size 16400M; 
  fastcgi_buffers 64 4K;
  client_body_temp_path /mnt/ext_drive/tmp/;

  location ~ \.php(?:$|/) {
        fastcgi_split_path_info ^(.+\.php)(/.+)$;
        include fastcgi_params;
        fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
        fastcgi_param PATH_INFO $fastcgi_path_info;
        fastcgi_param HTTPS on;
        fastcgi_param modHeadersAvailable true; #Avoid sending the security headers twice
        fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
        fastcgi_intercept_errors on;
        fastcgi_read_timeout 600;
        fastcgi_param htaccessWorking true;
  }

Best,
/ Ed

2

Sure that your PI is 64bit which is needed for anything above 4GB of upload size?

3

Just finding time to trying to resolve this...

Nope it is only a B model, but I previously had tried a 4G configuration in Nginx on both the Owncloud server and the reverse proxy.
I have now set both Pi's to 2000M and the file stops sync on a pdf.
Error shown is:

2016/09/06 21:53:28 [error] 32528#32528: *36 client intended to send too large body: 1391023 bytes,

Any further ideas guys?

4

http://nginx.org/en/support.html might be a good place to get further help with nginx.

5

Early days! However, I think I found the issue after following your lead and ended up reading this conversation.

I removed buffering from the reverse proxy server.
proxy_buffering off;
proxy_request_buffering off;

And, it has uploaded 90 of 1600 files -- so looking good!

Thanks for pointing me in the right direction R.Rancor.

6

The above did help with the proxy, but upon further investigation.

I was still getting a 413 error from the Nginx server with Owncloud running on it. :confounded:

With some digging I found this post which seems to have resolved my 413 and I have so far several GB of data with no errors.

To summarize the addition, I added the following in my server block:

  client_body_in_file_only clean;
  client_body_buffer_size 32K;

  # set max upload size
  client_max_body_size 4000M;

  sendfile on;
  send_timeout 600s;

For those run Owncloud via a Pacman Repo and need a fully-configured Nginx config file suitable for Owncloud, here is my full config.
N.B.: Owncloud is in a subfolder. Caching is configured with apcu and an aggressive country blacklist policy is in place with geoip. So be sure to find the relevant information for these on the Arch's wiki or remove the aforementioned sections from the config.

upstream php-handler {
      server 127.0.0.1:9000;
      #server unix:/run/php/php7.0-fpm.sock;
    }

server {
  listen 80;
  server_name 10.0.1.15 mysite.net www.mysite.net localhost;
  return 301 https://$server_name$request_uri;
}


server {

  if ($allowed_country = no) {
    return 443;
  }

  listen 443 ssl;
  server_name www.mysite.net;

  ssl_certificate /etc/nginx/keys/1_www.mysite.net_bundle.crt;
  ssl_certificate_key /etc/nginx/keys/1_www.mysite.net.key;

  root /usr/share/webapps/;

  index index.html index.php

  client_body_in_file_only clean;
  client_body_buffer_size 32K;

  # set max upload size
  client_max_body_size 4000M;

  sendfile on;
  send_timeout 600s;

  fastcgi_buffers 64 4K;
  client_body_temp_path /mnt/external_drive/;

  # Disable gzip to avoid the removal of the ETag header
  gzip off;

  # Add headers to serve security related headers
  add_header Strict-Transport-Security "max-age=15768000;    includeSubDomains; preload;";
  add_header X-Content-Type-Options nosniff;
  add_header X-Frame-Options "SAMEORIGIN";
  add_header X-XSS-Protection "1; mode=block";
  add_header X-Robots-Tag none;
  add_header X-Download-Options noopen;
  add_header X-Permitted-Cross-Domain-Policies none;


  location ~ ^/(?:\.htaccess|data|config|db_structure\.xml|README) {
     deny all;
  }

  location ~ ^/(build|tests|config|lib|3rdparty|templates|data)/ {
    deny all;
  }

  location ~ ^/(?:\.|autotest|occ|issue|indie|db_|console) {
    deny all;
  }


  rewrite ^/.well-known/carddav /remote.php/carddav/ permanent;
  rewrite ^/.well-known/caldav /remote.php/caldav/ permanent;

  error_page 403 /core/templates/403.php;
  error_page 404 /core/templates/404.php;

  location = /robots.txt {
     allow all;
     log_not_found off;
     access_log off;
  }


  location /owncloud {
     alias /usr/share/webapps/owncloud/;
     alias /usr/share/webapps/owncloud/;

     index index.html index.htm index.php;

     error_page 403 /owncloud/core/templates/403.php;
     error_page 404 /owncloud/core/templates/404.php;
     
     rewrite ^/owncloud/caldav(.*)$ /remote.php/caldav$1 redirect;
     rewrite ^/owncloud/carddav(.*)$ /remote.php/carddav$1 redirect;
     rewrite ^/owncloud/webdav(.*)$ /remote.php/carddav$1 redirect;
     rewrite ^(/owncloud/core/doc[^\/]+/)$ $1/index.html;
     # The following rules are only needed with webfinger
     rewrite ^/owncloud/.well-known/host-meta /public.php?service=host-meta last;
     rewrite ^/owncloud/.well-known/host-meta.json /public.php?service=host-meta-json last;
     rewrite ^/owncloud/.well-known/carddav /remote.php/carddav/  redirect;
     rewrite ^/owncloud/.well-known/caldav /remote.php/caldav/ redirect;
     try_files $uri $uri/ index.php;
 }

  location ~ \.php(?:$|/) {
      fastcgi_split_path_info ^(.+\.php)(/.+)$;
      include fastcgi_params;
      fastcgi_param SCRIPT_FILENAME     $document_root$fastcgi_script_name;
      fastcgi_param PATH_INFO $fastcgi_path_info;
      fastcgi_param HTTPS on;
      fastcgi_param modHeadersAvailable true; #Avoid sending the security headers twice
      fastcgi_pass unix:/run/php-fpm/php-fpm.sock;
      fastcgi_intercept_errors on;
      fastcgi_read_timeout 600;
      fastcgi_param htaccessWorking true;
      fastcgi_param PHP_VALUE "upload_max_filesize = 4000M \n  post_max_size=4000M";
  }

  # Adding the cache control header for js and css files
  # Make sure it is BELOW the location ~ \.php(?:$|/) { block
  location ~* \.(?:css|js)$ {
      add_header Cache-Control "public, max-age=7200";
      # Add headers to serve security related headers
      add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;";
      add_header X-Content-Type-Options nosniff;
      add_header X-Frame-Options "SAMEORIGIN";
      add_header X-XSS-Protection "1; mode=block";
      add_header X-Robots-Tag none;
      add_header X-Download-Options noopen;
      add_header X-Permitted-Cross-Domain-Policies none;
      # Optional: Don't log access to assets
      access_log off;
  }

  location / {
     index index.html index.php;
  }

  location ~* \.(?:svg|gif|png|html|ttf|woff|ico|jpg|jpeg)$ {
      try_files $uri /index.php$uri$is_args$args;
      # Optional: Don't log access to other assets
      access_log off;
  }
}
7

You forgot
client_max_body_size 10M; (or more, but at least 10MB, as clients are sending chunks of that size)

1 Like