nginx – WebServer: Send file that is still being written

I have a long-running process that creates large files for a web application that are stored in the file system.

Is there a way to deliver the download of this file via the web server (currently nginx) without ending the download if the file created up to that point has been downloaded and then waiting for the file to be created further?

At the moment there is always a wait until the file is complete, but therefore the download starts much later.

Kind Regards

NGINX reverse proxy not forwarding the request hostname to docker container

Problem:

We’ve setup a docker container running on port 3002 and then configured port 3002 to /path/ on my domain www.example.com. There’s an express rest api is running on 3002 port container which outputs the req.hostname and when I make a request from let’s say www.abc.com, the consoled value of req.hostname is shown to be www.example.com instead of www.abc.com.

Nginx Conf

server {
    listen 443 ssl;
    ssl_certificate /etc/ssl/__abc.crt;
    ssl_certificate_key /etc/ssl/abc.key;

    listen 80 default_server;
    listen [::]:80 default_server;
    location  / {
        proxy_pass http://localhost:3001/;
        proxy_set_header Host $host;
    }
    location /path/ {
        proxy_pass http://localhost:3002/;
        proxy_set_header Host $http_host;
    }
}

What changes do I have to make so I can get the www.abc.com in consoled value?

reverse proxy – Chaining multiple nginx error pages

I’m fighting a bit with nginx and try to get a “chain” of nginx error pages to work. Current state looks like this:

server {
    listen       443 ssl;
    server_name  ~^((?<repo>.*).)example.de$;

    ssl_certificate /etc/pki/tls/certs/cert.pem;
    ssl_certificate_key /etc/pki/tls/private/key.pem;
    client_max_body_size 100M;

    location / {
        proxy_pass http://backend_example/example/${repo}$request_uri;
        proxy_http_version 1.1;
        proxy_buffering off;
        proxy_connect_timeout 300;
        proxy_intercept_errors on;

        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_redirect off;

        rewrite  ^/$  /index.html  permanent;
        error_page 404 /backend_404.html;
    }

    location /backend_404.html {
        proxy_pass http://backend_example/example/${repo}/404.html;
        proxy_intercept_errors on;

        error_page 404 /error.html;
    }

    location /error.html {
        ssi on;
        internal;
        root /usr/share/nginx/html;
    }
}

What’s working:

  • if the file 404.html is available on the backend it will be delivered as intended
  • if the 404.html is not available on the backend I get the standard 404 from nginx instead of the local custom error.html
  • if I replace error_page 404 /backend_404.html; with error_page 404 /error.html; the error.html location also works

What I want to achieve:

  • if 404.html exists on the backend, deliver it
  • if 404.html not exist, deliver a custom error page error.html instead of the default one

API call err_connection_refused only on Chrome (NGINX backend)

I make API calls with a really really long parameter string (500 product ids):

https://www.mywebsite.com/xxxx/yyyy/load/?t=1590221086786&mode=cart&productIds%5B%5D=6413&productIds%5B%5D=6411&productIds%5B%5D=6410&productIds%5B%5D=6405&productIds%5B%5D=6402&productIds%5B%5D=6397&productIds%5B%5D=6396&productIds%5B%5D=6394&productIds%5B%5D=6392&productIds%5B%5D=6391&productIds%5B%5D=6390&productIds%5B%5D=6383&productIds%5B%5D=6379&productIds%5B%5D=6372&productIds%5B%5D=6371&productIds%5B%5D=6362&productIds%5B%5D=6361&productIds%5B%5D=6360&productIds%5B%5D=6359&productIds%5B%5D=6357&productIds%5B%5D=6356&productIds%5B%5D=6352&productIds%5B%5D=6351&productIds%5B%5D=6348&productIds%5B%5D=6346&productIds%5B%5D=6345&productIds%5B%5D=6342&productIds%5B%5D=6341&productIds%5B%5D=6340&productIds%5B%5D=6335&productIds%5B%5D=6332&productIds%5B%5D=6331&productIds%5B%5D=6330&productIds%5B%5D=6326&productIds%5B%5D=6322&productIds%5B%5D=6321&productIds%5B%5D=6319&productIds%5B%5D=6318&productIds%5B%5D=6317&productIds%5B%5D=6315&productIds%5B%5D=6312&productIds%5B%5D= …………..

This call is embedded in a page and is triggered during the DOM load.
Everything is served by NGINX over https.

When I load this page or try this call individually on IE, no problem it works fine.
When I try it in chrome it fails with a ERR_CONNECTION_CLOSED error.
When I try it in Firefox it fails without error message.

Any idea what nginx setting could fix that? Maybe related to compression deflate?
In Apache I don’t have the issue.

domain name system – How do I add hostnames to nginx so they are externally accessible?

Let’s say I have a domain name example.com and want to add a site, like admin.example.com. In nginx I can then set server_name in the configuration of this site to admin.example.com. However, if I try to access the site, it cannot find the IP address.

Do I need to add this to the DNS record itself? Isn’t it possible to make the server itself the DNS server, or to just point all the hostname/subdomains (not sure what the right term is, but the * part in *.example.com) to the servers’ IP in the original DNS record? If I want to add another hostname like admin2.example.com I don’t want to edit the DNS record again.

I’m sure I’m missing something obvious here. I also can’t find any tutorial on this, which I find weird, as it should be something which is rather common.

php – Nginx proxy request to url specified in GET variable

I need Nginx to respond to a request like

https://example.org/proxy/?url=https://othersite.com/path/file.php%%a=test123%%b=321tset

or a similar method, like

https://example.org/proxy/https://othersite.com/path/file.php?a=test123&b=321tset

by proxying the request to

https://othersite.com/path/file.php?a=test123&b=321tset

Is there a way to do this with rewriting or a different rule? Any help would be appreciated. Thank you in advance.

cors – Nginx Cross-Origin Request Blocked

I have this error in my web page:
“Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://www.cdn-example.com. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).”

I have Nginx and this is my configuration:
/etc/nginx/nginx.conf

user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;

events {
    worker_connections 768;
    # multi_accept on;
}

http {

    ##
    # Basic Settings
    ##

    sendfile on;
    tcp_nopush on;
    tcp_nodelay on;
    keepalive_timeout 65;
    types_hash_max_size 2048;
    # server_tokens off;

    # server_names_hash_bucket_size 64;
    # server_name_in_redirect off;

    include /etc/nginx/mime.types;
    default_type application/octet-stream;

    ##
    # SSL Settings
    ##

    ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
    ssl_prefer_server_ciphers on;

    ##
    # Logging Settings
    ##

    access_log /var/log/nginx/access.log;
    error_log /var/log/nginx/error.log;

    ##
    # Gzip Settings
    ##

    gzip on;

    # gzip_vary on;
    # gzip_proxied any;
    # gzip_comp_level 6;
    # gzip_buffers 16 8k;
    # gzip_http_version 1.1;
    # gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

    ##
    # Virtual Host Configs
    ##

    include /etc/nginx/conf.d/*.conf;
    include /etc/nginx/sites-enabled/*;
}

/etc/nginx/sites-enabled/default

server {
    listen 80;
    listen (::):80;
    return 301 https://$host$request_uri;
}

server {
   # listen 443 default_server;
   # listen            (::):443;
    listen 443 default_server;
    listen (::):443;
    ssl on;
    ssl_certificate /etc/ssl/certs/pem-access.pem;
    ssl_certificate_key /etc/ssl/private/key-access.key;
    access_log            /var/log/nginx/nginx.access.log;
    error_log            /var/log/nginx/nginx.error.log;
    location / {
        add_header 'Access-Control-Allow-Origin' "https://www.cdn-example.com" always;
    }
}

Mi original site is other https://www.example.com

nginx – 301 redirect requests that are not proxied

I’ve got a WordPress app running on Apache (on blog.example.com), and an Nginx server that works as a reverse proxy (on example.com). My main goal is to have blog.example.com accessible through example.com/blog. I’ve achieved this.

My secondary goal is to have blog.example.com not accessible anymore, and to redirect requests directly to blog.example.com to example.com/blog. This I haven’t been able to achieve.

My idea was to use X-Forwarded-Host headers for this, but a) I don’t know if this is the right approach, and b) I can’t get this to work. My setup is as follows:

Nginx config:

rewrite ^/blog$ https://example.com/blog/ permanent;
location ^~ /blog/ {
    proxy_set_header Forwarded dev.example.com;
    proxy_set_header Host blog.example.com;
    proxy_pass https://blog.example.com/;
}

Apache config:

<IfModule mod_rewrite.c>
  RewriteEngine On
  RewriteCond %{HTTP:X-Forwarded-Host} !^dev.example.com$ (NC)
  RewriteRule /?(.*)$ https://example.com/blog/$1 (R=301,L)
</IfModule>

This setup correctly performs a reverse proxy (aka, if I go to example.com/blog, I’ll see the contents of blog.example.com). However if I go to blog.example.com, I’m not redirected. Furthermore I don’t know how to debug this, as the X-Forwarded-Host header is not visible in any web development tools I use as far as I know.

Any and all help is welcome, thanks in advance!

LARAVEL ELASTICBEANSTALK NGINX MODIFICAR CONFIGURACIÓN

estoy subiendo mi aplicación laravel a nginx, de elasticbeanstalk de amazon, cuando lo subo no funcionan las rutas /api/*, en la documentación de Laravel especifica que hay que añadir lo siguiente a la configuración de nginx:

location / { try_files $uri $uri/ /index.php?$query_string; }

Al añadirlo funciona, pero el problema viene al cargar una nueva versión, al cargar la nueva versión el archivo de configuración de nginx vuelve a estar como antes, sin esa linea.

He probado también a añadir lo siguiente es las .ebextensions, pero no funciona

    container_commands:
      stop_nginx:
        command: "sudo service nginx stop"
      crear_nginx_config:
        command: "sudo cp /etc/nginx/nginx.complete.conf /etc/nginx/nginx.conf"
        leader_only: true
      start_nginx:
        command: "sudo service nginx start"
        leader_only: true

Donde nginx.complete.conf es un archivo con toda la configuración inicial mas la linea de antes. No entiendo mucho de los archivos de .ebextensions, y desconozco si estan bien configurados. Gracias de antemano

mysql – WordPress website hosted on nginx ubuntu isn’t loading anymore

I just found out the WordPress website isn’t running anymore. When opening example.in, it simply shows the text Error establishing a database connection. The wp website is the folder /var/www/examplewp

I have other non-PHP based websites running smoothly on the same server. Even the xxx.example.in which is a non-php based website is working.

I tried opening files such as example.in/readmore.html or example.in/hello.txt which I created in the base folder of the WP and that’s working.

Here’s the details:

php -v

PHP 7.2.19-0ubuntu0.18.10.1 (cli) (built: Jun  4 2019 14:46:43) ( NTS )
Copyright (c) 1997-2018 The PHP Group
Zend Engine v3.2.0, Copyright (c) 1998-2018 Zend Technologies
with Zend OPcache v7.2.19-0ubuntu0.18.10.1, Copyright (c) 1999-2018, by Zend Technologies

uname -a

Linux ubuntu-s-1vcpu-1gb-blr1-01 4.18.0-25-generic #26-Ubuntu SMP Mon Jun 24 09:32:08 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

This is my xxx_nginx.conf file

server {
        server_name example.in www.example.in;
        root /var/www/examplewp;
        access_log /var/log/nginx/wp_client_access.log;
        error_log /var/log/nginx/wp_client_error.log;

        location / {
                index   index.php index.html;
                #try_files      $uri $uri/ /index.php?$args;
        }
        # Specify a charset
        charset                         utf-8;
        # GZIP
        gzip                            off;

        # Add trailing slash to */wp-admin requests.
        rewrite /wp-admin$ $scheme://$host$uri/ permanent;

        # Prevents hidden files (beginning with a period) from being served
        location ~ /. {
                access_log                      off;
                log_not_found                   off;
                deny                            all;
        }
        ###########
        # SEND EXPIRES HEADERS AND TURN OFF 404 LOGGING
        ###########

        location ~* ^.+.(xml|ogg|ogv|svg|svgz|eot|otf|woff|mp4|ttf|css|rss|atom|js|jpg|jpeg|gif|png|ico|zip|tgz|gz|rar|bz2|doc|xls|exe|ppt|tar|mid|midi|wav|bmp|rtf)$ {
                access_log                      off;
                log_not_found                   off;
                expires                         max;
        }

        # Pass all .php files onto a php-fpm or php-cgi server
        location ~ .php$ {
                try_files                       $uri =404;
                include                         /etc/nginx/fastcgi_params;
                fastcgi_read_timeout            3600s;
                fastcgi_buffer_size             128k;
                fastcgi_buffers                 4 128k;
                fastcgi_param                   SCRIPT_FILENAME $document_root$fastcgi_script_name;
                fastcgi_pass                    unix:/run/php/php7.2-fpm.sock;
                fastcgi_pass                    unix:/run/php/php7.2-fpm.sock;
                fastcgi_index                   index.php;
        }

        # ROBOTS

         location = /robots.txt {
               allow all;
               log_not_found off;
               access_log off;
        }
        # RESTRICTIONS
        location ~* /(?:uploads|files)/.*.php$ {
                deny all;
        }

    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/example.in/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/example.in/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}

server {
    if ($host = www.example.in) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


    if ($host = example.in) {
        return 301 https://$host$request_uri;
    } # managed by Certbot

        server_name example.in www.example.in;
    listen 80;
    return 404; # managed by Certbot
}

I tried restarting the server with sudo service nginx restart but it doesn’t help. Even the HTML or TXT files aren’t opening. I tried with the command line sudo service php7-fpm restart but got the response:

Failed to restart php7-fpm.service: Unit php7-fpm.service not found.

I can open info.php file though which has the phpinfo(); and see all the PHP related files.

I also check the error log but it’s empty: /var/log/nginx/wp_client_error.log

I tried restarting the mysql with this command land: sudo /etc/init.d/mysql start and got the following error:

(....) Starting mysql (via systemctl): mysql.serviceJob for mysql.service failed because the control process exited with error code.
See "systemctl status mysql.service" and "journalctl -xe" for details.
 **failed!**