Commit graph

6496 commits

Author SHA1 Message Date
Maxim Dounin
95e1c4e8cf Win32: properly enabled select on Windows.
Previously, select was compiled in by default, but the NGX_HAVE_SELECT
macro was not set, resulting in iocp being used by default unless
the "--with-select_module" configure option was explicitly specified.
Since the iocp module is not finished and does not work properly, this
effectively meant that the "--with-select_module" option was mandatory.

With the change NGX_HAVE_SELECT is properly set, making "--with-select_module"
optional.  Accordingly, it is removed from misc/GNUmakefile win32 target.
2019-01-24 21:51:00 +03:00
Maxim Dounin
299f677cd7 Events: fixed copying of old events in poll init.
Previously, the code incorrectly assumed "ngx_event_t *" elements
instead of "struct pollfd".

This is mostly cosmetic change, as this code is never called now.
2019-01-24 21:50:37 +03:00
Roman Arutyunyan
7618eefb58 Stream: do not split datagrams when limiting proxy rate.
Previously, when using proxy_upload_rate and proxy_download_rate, the buffer
size for reading from a socket could be reduced as a result of rate limiting.
For connection-oriented protocols this behavior is normal since unread data will
normally be read at the next iteration.  But for datagram-oriented protocols
this is not the case, and unread part of the datagram is lost.

Now buffer size is not limited for datagrams.  Rate limiting still works in this
case by delaying the next reading event.
2018-12-27 19:37:34 +03:00
Roman Arutyunyan
8a9615771f Prevented scheduling events on a shared connection.
A shared connection does not own its file descriptor, which means that
ngx_handle_read_event/ngx_handle_write_event calls should do nothing for it.
Currently the c->shared flag is checked in several places in the stream proxy
module prior to calling these functions.  However it was not done everywhere.
Missing checks could lead to calling
ngx_handle_read_event/ngx_handle_write_event on shared connections.

The problem manifested itself when using proxy_upload_rate and resulted in
either duplicate file descriptor error (e.g. with epoll) or incorrect further
udp packet processing (e.g. with kqueue).

The fix is to set and reset the event active flag in a way that prevents
ngx_handle_read_event/ngx_handle_write_event from scheduling socket events.
2019-01-14 20:36:23 +03:00
Vladimir Homutov
33a22c1cfa Added the ngx_http_test_required_predicates() function.
In contrast to ngx_http_test_predicates(), it requires all values to be
non-empty and not equal to "0".
2019-01-17 14:31:04 +03:00
Vladimir Homutov
22b7c3d174 Version bump. 2019-01-17 14:31:01 +03:00
Maxim Dounin
66b4aba566 release-1.15.8 tag 2018-12-25 17:53:03 +03:00
Maxim Dounin
a1ef8b9c05 nginx-1.15.8-RELEASE 2018-12-25 17:53:03 +03:00
Vladimir Homutov
2f8c01b2e3 Autoindex: fixed possible integer overflow on 32-bit systems. 2018-12-25 12:59:24 +03:00
Gena Makhomed
d4f6bf35cb Contrib: vim syntax, update core and 3rd party module directives. 2018-12-24 16:30:10 +02:00
Maxim Dounin
9d7d919697 Win32: removed NGX_DIR_MASK concept.
Previous interface of ngx_open_dir() assumed that passed directory name
has a room for NGX_DIR_MASK at the end (NGX_DIR_MASK_LEN bytes).  While all
direct users of ngx_dir_open() followed this interface, this also implied
similar requirements for indirect uses - in particular, via ngx_walk_tree().

Currently none of ngx_walk_tree() uses provides appropriate space, and
fixing this does not look like a right way to go.  Instead, ngx_dir_open()
interface was changed to not require any additional space and use
appropriate allocations instead.
2018-12-24 21:07:05 +03:00
Sergey Kandaurov
cb539b46a4 Userid: using stub for AF_UNIX addresses.
Previously, AF_UNIX addresses misbehaved as AF_INET, which typically resulted
in $uid_set composed from the middle of sun_path.
2018-12-24 19:55:00 +03:00
Sergey Kandaurov
1c27bfd85e SSL: avoid reading on pending SSL_write_early_data().
If SSL_write_early_data() returned SSL_ERROR_WANT_WRITE, stop further reading
using a newly introduced c->ssl->write_blocked flag, as otherwise this would
result in SSL error "ssl3_write_bytes:bad length".  Eventually, normal reading
will be restored by read event posted from successful SSL_write_early_data().

While here, place "SSL_write_early_data: want write" debug on the path.
2018-12-18 15:15:15 +03:00
Maxim Dounin
90fa22cb44 Geo: fixed handling of AF_UNIX client addresses (ticket #1684).
Previously, AF_UNIX client addresses were handled as AF_INET, leading
to unexpected results.
2018-12-14 18:11:06 +03:00
Ruslan Ermilov
6a66ebfda3 Upstream: implemented $upstream_bytes_sent. 2018-12-13 17:23:07 +03:00
Roman Arutyunyan
8ad77cc254 Resolver: report SRV resolve failure if all A resolves failed.
Previously, if an SRV record was successfully resolved, but all of its A
records failed to resolve, NXDOMAIN was returned to the caller, which is
considered a successful resolve rather than an error.  This could result in
losing the result of a previous successful resolve by the caller.

Now NXDOMAIN is only returned if at least one A resolve completed with this
code.  Otherwise the error state of the first A resolve is returned.
2018-12-11 19:41:22 +03:00
Roman Arutyunyan
a9c10e9571 Copy regex unnamed captures to cloned subrequests.
Previously, unnamed regex captures matched in the parent request, were not
available in a cloned subrequest.  Now 3 fields related to unnamed captures
are copied to a cloned subrequest: r->ncaptures, r->captures and
r->captures_data.  Since r->captures cannot be changed by either request after
creating a clone, a new flag r->realloc_captures is introduced to force
reallocation of r->captures.

The issue was reported as a proxy_cache_background_update misbehavior in
http://mailman.nginx.org/pipermail/nginx/2018-December/057251.html.
2018-12-11 13:09:00 +03:00
Roman Arutyunyan
33fd6c222a Version bump. 2018-12-11 13:12:35 +03:00
Maxim Dounin
7c70c30027 release-1.15.7 tag 2018-11-27 17:40:21 +03:00
Maxim Dounin
347e47ed16 nginx-1.15.7-RELEASE 2018-11-27 17:40:20 +03:00
Maxim Dounin
ba9b0f3f29 Updated OpenSSL used for win32 builds. 2018-11-27 17:02:56 +03:00
Maxim Dounin
f9cb4cb6cb Negative size buffers detection.
In the past, there were several security issues which resulted in
worker process memory disclosure due to buffers with negative size.
It looks reasonable to check for such buffers in various places,
much like we already check for zero size buffers.

While here, removed "#if 1 / #endif" around zero size buffer checks.
It looks highly unlikely that we'll disable these checks anytime soon.
2018-11-26 18:29:56 +03:00
Maxim Dounin
3272b66121 Mp4: fixed possible pointer overflow on 32-bit platforms.
On 32-bit platforms mp4->buffer_pos might overflow when a large
enough (close to 4 gigabytes) atom is being skipped, resulting in
incorrect memory addesses being read further in the code.  In most
cases this results in harmless errors being logged, though may also
result in a segmentation fault if hitting unmapped pages.

To address this, ngx_mp4_atom_next() now only increments mp4->buffer_pos
up to mp4->buffer_end.  This ensures that overflow cannot happen.
2018-11-21 20:23:16 +03:00
Maxim Dounin
6123ec0dd1 Limit req: "delay=" parameter.
This parameter specifies an additional "soft" burst limit at which requests
become delayed (but not yet rejected as it happens if "burst=" limit is
exceeded).  Defaults to 0, i.e., all excess requests are delayed.

Originally inspired by Vladislav Shabanov
(http://mailman.nginx.org/pipermail/nginx-devel/2016-April/008126.html).
Further improved based on a patch by Peter Shchuchkin
(http://mailman.nginx.org/pipermail/nginx-devel/2018-October/011522.html).
2018-11-21 18:56:50 +03:00
Maxim Dounin
a703fca82c Limit req: fixed error message wording. 2018-11-21 18:56:44 +03:00
Vladimir Homutov
4f05b2b687 Upstream: revised upstream response time variables.
Variables now do not depend on presence of the HTTP status code in response.
If the corresponding event occurred, variables contain time between request
creation and the event, and "-" otherwise.

Previously, intermediate value of the $upstream_response_time variable held
unix timestamp.
2018-11-21 13:40:40 +03:00
Vladimir Homutov
14a5fc247e Upstream: removed unused ngx_http_upstream_t.timeout field. 2018-11-21 13:40:36 +03:00
Maxim Dounin
8aa80bbf93 Core: ngx_explicit_memzero(). 2018-11-15 21:28:02 +03:00
Ruslan Ermilov
9b3272537f Core: free shared memory on cycle initialization failure. 2018-11-15 15:28:54 +03:00
Vladimir Homutov
7f83ab29f9 Stream: proxy_requests directive.
The directive allows to drop binding between a client and existing UDP stream
session after receiving a specified number of packets.  First packet from the
same client address and port will start a new session.  Old session continues
to exist and will terminate at moment defined by configuration: either after
receiving the expected number of responses, or after timeout, as specified by
the "proxy_responses" and/or "proxy_timeout" directives.

By default, proxy_requests is zero (disabled).
2018-11-12 16:29:30 +03:00
Vladimir Homutov
7d1cc21003 Stream: session completion check code moved to a separate function.
The code refactored to simplify the ngx_stream_proxy_process() function
and facilitate adding new session termination conditions.
2018-11-12 12:05:03 +03:00
Vladimir Homutov
89d29aba13 Stream: fixed possible use of a freed connection.
The session handler may result in session termination, thus a connection
pool (from which c->udp was allocated) may be destroyed.
2018-11-07 13:22:14 +03:00
Vladimir Homutov
f4572c2e32 Version bump. 2018-11-13 14:42:47 +03:00
Maxim Dounin
5af37b261d release-1.15.6 tag 2018-11-06 16:32:09 +03:00
Maxim Dounin
82bdfe5c07 nginx-1.15.6-RELEASE 2018-11-06 16:32:08 +03:00
Maxim Dounin
5aaad3b54b gRPC: limited allocations due to ping and settings frames. 2018-11-06 16:29:59 +03:00
Ruslan Ermilov
81c59db62e HTTP/2: limit the number of idle state switches.
An attack that continuously switches HTTP/2 connection between
idle and active states can result in excessive CPU usage.
This is because when a connection switches to the idle state,
all of its memory pool caches are freed.

This change limits the maximum allowed number of idle state
switches to 10 * http2_max_requests (i.e., 10000 by default).
This limits possible CPU usage in one connection, and also
imposes a limit on the maximum lifetime of a connection.

Initially reported by Gal Goldshtein from F5 Networks.
2018-11-06 16:29:49 +03:00
Ruslan Ermilov
b864af3342 HTTP/2: flood detection.
Fixed uncontrolled memory growth in case peer is flooding us with
some frames (e.g., SETTINGS and PING) and doesn't read data.  Fix
is to limit the number of allocated control frames.
2018-11-06 16:29:35 +03:00
Roman Arutyunyan
0cbcdb3423 Mp4: fixed reading 64-bit atoms.
Previously there was no validation for the size of a 64-bit atom
in an mp4 file.  This could lead to a CPU hog when the size is 0,
or various other problems due to integer underflow when calculating
atom data size, including segmentation fault or worker process
memory disclosure.
2018-11-06 16:29:18 +03:00
Maxim Dounin
c18c4970c3 Cache: improved keys zone size error reporting.
After this change, too small keys zones are explicitly reported as such,
much like in the other modules which use shared memory.
2018-10-31 16:49:40 +03:00
Maxim Dounin
e63f0258b7 Cache: fixed minimum cache keys zone size limit.
Size of a shared memory zones must be at least two pages - one page
for slab allocator internal data, and another page for actual allocations.
Using 8192 instead is wrong, as there are systems with page sizes other
than 4096.

Note well that two pages is usually too low as well.  In particular, cache
is likely to use two allocations of different sizes for global structures,
and at least four pages will be needed to properly allocate cache nodes.
Except in a few very special cases, with keys zone of just two pages nginx
won't be able to start.  Other uses of shared memory impose a limit
of 8 pages, which provides some room for global allocations.  This patch
doesn't try to address this though.

Inspired by ticket #1665.
2018-10-31 16:49:39 +03:00
chronolaw
f2baff308d A minor code clean for macro ngx_event_get_conf in ngx_event.h. 2018-10-19 13:50:36 +08:00
Maxim Dounin
0e1c908577 SSL: explicitly set maximum version (ticket #1654).
With maximum version explicitly set, TLSv1.3 will not be unexpectedly
enabled if nginx compiled with OpenSSL 1.1.0 (without TLSv1.3 support)
will be run with OpenSSL 1.1.1 (with TLSv1.3 support).
2018-10-23 22:11:48 +03:00
Vladimir Homutov
a74dfd3ad8 Upstream: proxy_socket_keepalive and friends.
The directives enable the use of the SO_KEEPALIVE option on
upstream connections.  By default, the value is left unchanged.
2018-10-03 14:08:51 +03:00
Vladimir Homutov
2a14738076 Version bump. 2018-10-03 17:02:44 +03:00
Maxim Dounin
b5094a3411 release-1.15.5 tag 2018-10-02 18:13:52 +03:00
Maxim Dounin
84fee2220c nginx-1.15.5-RELEASE 2018-10-02 18:13:51 +03:00
Maxim Dounin
836236474f SSL: fixed segfault on renegotiation (ticket #1646).
In e3ba4026c02d (1.15.4) nginx own renegotiation checks were disabled
if SSL_OP_NO_RENEGOTIATION is available.  But since SSL_OP_NO_RENEGOTIATION
is only set on a connection, not in an SSL context, SSL_clear_option()
removed it as long as a matching virtual server was found.  This resulted
in a segmentation fault similar to the one fixed in a6902a941279 (1.9.8),
affecting nginx built with OpenSSL 1.1.0h or higher.

To fix this, SSL_OP_NO_RENEGOTIATION is now explicitly set in
ngx_http_ssl_servername() after adjusting options.  Additionally, instead
of c->ssl->renegotiation we now check c->ssl->handshaked, which seems
to be a more correct flag to test, and will prevent the segmentation fault
from happening even if SSL_OP_NO_RENEGOTIATION is not working.
2018-10-02 17:46:18 +03:00
Ruslan Ermilov
318d5841be Fixed off-by-one error in shared zone initialization.
On systems without atomic ops, not enough space was allocated
for mutex's file name during shared zone initialization.
2018-10-02 13:32:52 +03:00
Ruslan Ermilov
7d49e3424e SSL: fixed unlocked access to sess_id->len. 2018-09-25 14:07:59 +03:00