Struct ClientBuilder 

pub struct ClientBuilder {

    github_token: Option<SecretString>,
    remaps: Option<Remaps>,
    fallback_extensions: Vec<String>,
    index_files: Option<Vec<String>>,
    includes: Option<RegexSet>,
    excludes: Option<RegexSet>,
    exclude_all_private: bool,
    exclude_private_ips: bool,
    exclude_link_local_ips: bool,
    exclude_loopback_ips: bool,
    include_mail: bool,
    max_redirects: usize,
    max_retries: u64,
    min_tls_version: Option<Version>,
    user_agent: String,
    allow_insecure: bool,
    schemes: HashSet<String>,
    custom_headers: HeaderMap,
    method: Method,
    accepted: HashSet<StatusCode>,
    timeout: Option<Duration>,
    base: Option<Base>,
    retry_wait_time: Duration,
    require_https: bool,
    cookie_jar: Option<Arc<CookieStoreMutex>>,
    include_fragments: bool,
    include_wikilinks: bool,
    plugin_request_chain: RequestChain,
    rate_limit_config: RateLimitConfig,
    hosts: HostConfigs,
}

Builder for Client.

See crate-level documentation for usage example.

Fields

github_token: Option<SecretString>

Optional GitHub token used for GitHub links.

This allows much more request before getting rate-limited.

Rate-limiting Defaults

As of Feb 2022, it’s 60 per hour without GitHub token v.s. 5000 per hour with token.

remaps: Option<Remaps>

Remap URIs matching a pattern to a different URI.

This makes it possible to remap any HTTP/HTTPS endpoint to a different HTTP/HTTPS one. This feature could also be used to proxy certain requests.

Usage Notes

Use with caution because a large set of remapping rules may cause performance issues.

Furthermore rules are executed sequentially and multiple mappings for the same URI are allowed, so it is up to the library user’s discretion to make sure rules don’t conflict with each other.

fallback_extensions: Vec<String>

Automatically append file extensions to file:// URIs as needed

This option takes effect on file:// URIs which do not exist.

index_files: Option<Vec<String>>

Index file names to use when resolving file:// URIs which point to directories.

For local directory links, if this is non-None, then at least one index file from this list must exist in order for the link to be considered valid. Index files names are required to match regular files, aside from the special . name which will match the directory itself.

If None, index file checking is disabled and directory links are valid as long as the directory exists on disk.

In the ClientBuilder, this defaults to None.

includes: Option<RegexSet>

Links matching this set of regular expressions are always checked.

This has higher precedence over ClientBuilder::excludes, but has lower precedence compared to any other exclude_ fields or ClientBuilder::schemes below.

excludes: Option<RegexSet>

Links matching this set of regular expressions are ignored, except when a link also matches against ClientBuilder::includes.

exclude_all_private: bool

When true, exclude all private network addresses.

This effectively turns on the following fields:

• ClientBuilder::exclude_private_ips
• ClientBuilder::exclude_link_local_ips
• ClientBuilder::exclude_loopback_ips

exclude_private_ips: bool

When true, exclude private IP addresses.

IPv4

The private address ranges are defined in IETF RFC 1918 and include:

• 10.0.0.0/8
• 172.16.0.0/12
• 192.168.0.0/16

IPv6

The address is a unique local address (fc00::/7).

This property is defined in IETF RFC 4193.

Note

Unicast site-local network was defined in IETF RFC 4291, but was fully deprecated in IETF RFC 3879. So it is NOT considered as private on this purpose.

exclude_link_local_ips: bool

When true, exclude link-local IPs.

IPv4

The address is 169.254.0.0/16.

This property is defined by IETF RFC 3927.

IPv6

The address is a unicast address with link-local scope, as defined in RFC 4291.

A unicast address has link-local scope if it has the prefix fe80::/10, as per RFC 4291 section 2.4.

exclude_loopback_ips: bool

When true, exclude loopback IP addresses.

IPv4

This is a loopback address (127.0.0.0/8).

This property is defined by IETF RFC 1122.

IPv6

This is the loopback address (::1), as defined in IETF RFC 4291 section 2.5.3.

include_mail: bool

When true, check mail addresses.

max_redirects: usize

Maximum number of redirects per request before returning an error.

Defaults to DEFAULT_MAX_REDIRECTS.

max_retries: u64

Maximum number of retries per request before returning an error.

Defaults to DEFAULT_MAX_RETRIES.

min_tls_version: Option<Version>

Minimum accepted TLS version.

user_agent: String

User-agent used for checking links.

Defaults to DEFAULT_USER_AGENT.

Notes

This may be helpful for bypassing certain firewalls.

allow_insecure: bool

When true, accept invalid SSL certificates.

Warning

You should think very carefully before allowing invalid SSL certificates. It will accept any certificate for any site to be trusted including expired certificates. This introduces significant vulnerabilities, and should only be used as a last resort.

schemes: HashSet<String>

Set of accepted URL schemes.

Only links with matched URI schemes are checked. This has no effect when it’s empty.

custom_headers: HeaderMap

Default headers for every request.

This allows working around validation issues on some websites. See also here for usage examples.

method: Method

HTTP method used for requests, e.g. GET or HEAD.

accepted: HashSet<StatusCode>

Set of accepted return codes / status codes.

Unmatched return codes/ status codes are deemed as errors.

timeout: Option<Duration>

Response timeout per request.

base: Option<Base>

Base for resolving paths.

E.g. if the base is /home/user/ and the path is file.txt, the resolved path would be /home/user/file.txt.

retry_wait_time: Duration

Initial time between retries of failed requests.

Defaults to DEFAULT_RETRY_WAIT_TIME_SECS.

Notes

For each request, the wait time increases using an exponential backoff mechanism. For example, if the value is 1 second, then it waits for 2 ^ (N-1) seconds before the N-th retry.

This prevents spending too much system resources on slow responders and prioritizes other requests.

require_https: bool

When true, requires using HTTPS when it’s available.

This would treat unencrypted links as errors when HTTPS is available. It has no effect on non-HTTP schemes or if the URL doesn’t support HTTPS.

cookie_jar: Option<Arc<CookieStoreMutex>>

Cookie store used for requests.

See https://docs.rs/reqwest/latest/reqwest/struct.ClientBuilder.html#method.cookie_store

include_fragments: bool

Enable the checking of fragments in links.

include_wikilinks: bool

Enable the checking of wikilinks in markdown files. Note that base must not be None if you set this true.

plugin_request_chain: RequestChain

Requests run through this chain where each item in the chain can modify the request. A chained item can also decide to exit early and return a status, so that subsequent chain items are skipped and the lychee-internal request chain is not activated.

rate_limit_config: RateLimitConfig

Global rate limiting configuration that applies as defaults to all hosts

hosts: HostConfigs

Per-host configuration overrides

Implementations

impl ClientBuilder

pub fn builder() -> ClientBuilderBuilder<((), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), (), ())>

Create a builder for building ClientBuilder. On the builder, call .github_token(...)(optional), .remaps(...)(optional), .fallback_extensions(...)(optional), .index_files(...)(optional), .includes(...)(optional), .excludes(...)(optional), .exclude_all_private(...)(optional), .exclude_private_ips(...)(optional), .exclude_link_local_ips(...)(optional), .exclude_loopback_ips(...)(optional), .include_mail(...)(optional), .max_redirects(...)(optional), .max_retries(...)(optional), .min_tls_version(...)(optional), .user_agent(...)(optional), .allow_insecure(...)(optional), .schemes(...)(optional), .custom_headers(...)(optional), .method(...)(optional), .accepted(...)(optional), .timeout(...)(optional), .base(...)(optional), .retry_wait_time(...)(optional), .require_https(...)(optional), .cookie_jar(...)(optional), .include_fragments(...)(optional), .include_wikilinks(...)(optional), .plugin_request_chain(...)(optional), .rate_limit_config(...)(optional), .hosts(...)(optional) to set the values of the fields. Finally, call .build() to create the instance of ClientBuilder.

impl ClientBuilder

pub fn client(self) -> Result<Client>

Instantiates a Client.

Errors

Returns an Err if:

• The user-agent contains characters other than ASCII 32-127.
• The reqwest client cannot be instantiated. This occurs if a TLS backend cannot be initialized or the resolver fails to load the system configuration. See here.
• The GitHub client cannot be created. Since the implementation also uses reqwest under the hood, this errors in the same circumstances as the last one.

fn build_host_clients( &self, redirect_history: &RedirectHistory, ) -> Result<ClientMap>

Build the host-specific clients with their host-specific headers

fn build_client( &self, redirect_history: &RedirectHistory, ) -> Result<ClientBuilder>

Create a reqwest::ClientBuilder based on various fields

fn default_headers(&self) -> Result<HeaderMap>

Trait Implementations

impl Clone for ClientBuilder

fn clone(&self) -> ClientBuilder

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for ClientBuilder

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Default for ClientBuilder

fn default() -> Self

Returns the “default value” for a type.