From fd317f2af4b947d8bf516d6f4337585f52b5632d Mon Sep 17 00:00:00 2001 From: Moises Macia Date: Tue, 26 May 2026 21:10:54 +0000 Subject: [PATCH] add docs --- lib/feed_reader/transport.ex | 44 ++++++++++++++++++++++++++++++++---- 1 file changed, 40 insertions(+), 4 deletions(-) diff --git a/lib/feed_reader/transport.ex b/lib/feed_reader/transport.ex index 6c26265..c941fe7 100644 --- a/lib/feed_reader/transport.ex +++ b/lib/feed_reader/transport.ex @@ -2,9 +2,44 @@ defmodule FeedReader.Transport do @moduledoc """ Injectable HTTP transport boundary used by remote feed operations. - Implementations return raw response chunks as an enumerable. The response is - intentionally not decompressed here; the download layer owns decoded-body - checksums and decompression. + Implementations return raw response chunks as an enumerable. They must not + automatically decompress the response: the downloader owns decompression, + decoded-byte accounting, and checksums. `final_url` must include redirects; + a 304 response has an empty body and is returned without downloading content. + + `request/2` receives the full option list passed to the remote downloader. + The default `FeedReader.ReqTransport` recognizes these transport options: + + - `:http_client` — module implementing this behaviour, selected by + `Transport.request/2`; defaults to `FeedReader.ReqTransport`. The selected + module receives the full options list, including this option. + - `:http_options` — Req request options forwarded to `Req.run/2`. The transport + enforces streamed raw bodies, disables automatic decompression, and sets the + retry and redirect options described below; redirects are limited to five. + A custom `:headers` list replaces the defaults. Otherwise, headers default to + `user-agent: feedex/1` and `accept-encoding: gzip, deflate, identity`. + - `:previous_etag` — if binary, sent verbatim as `If-None-Match` on the + GET request. It replaces any existing `If-None-Match` header. A 304 response + returns `{:ok, response}` with `status: 304` and `body: []`; no HEAD request + is made. + - `:http_retry_attempts` — non-negative number of retries after the initial + request; defaults to `5`. The Req transport retries status 400 and 500 and + most transport errors. TLS alerts and redirect failures are terminal. + - `:retry_delay` — non-negative delay in milliseconds or an arity-one Req + retry-delay function; defaults to `0`. Invalid values use the default. + + The downloader also passes these recognized options through to custom + transports, but consumes them itself rather than using them to configure HTTP: + + - `:previous_checksum` — decoded-body checksum used for the unchanged result. + - `:max_compressed_bytes` — maximum total raw response bytes, default 50 MiB. + - `:max_decoded_bytes` — maximum total decompressed bytes, default 50 MiB. + - `:max_temp_bytes` — maximum bytes written to the owned temporary file, + default 50 MiB. + - `:max_expansion_ratio` — maximum decoded-to-compressed byte ratio, default + `100`. + + Other options remain available to injected transport implementations. """ @type response :: %{ @@ -14,10 +49,11 @@ defmodule FeedReader.Transport do required(:body) => Enumerable.t() } + @doc "Performs a GET and returns raw response chunks and response metadata." @callback request(url :: String.t(), opts :: keyword()) :: {:ok, response()} | {:error, term()} - @doc "Dispatches a request to the configured or default HTTP client." + @doc "Dispatches the request to `:http_client`, or `FeedReader.ReqTransport` by default." @spec request(String.t(), keyword()) :: {:ok, response()} | {:error, term()} def request(url, opts \\ []) do client = Keyword.get(opts, :http_client, FeedReader.ReqTransport)