Struct WebsiteChecker 

pub(crate) struct WebsiteChecker {
    method: Method,
    github_client: Option<Octocrab>,
    plugin_request_chain: RequestChain,
    max_retries: u64,
    retry_wait_time: Duration,
    accepted: HashSet<StatusCode>,
    require_https: bool,
    include_fragments: bool,
    fragment_checker: FragmentChecker,
    redirect_history: RedirectHistory,
    host_pool: Arc<HostPool>,
}

Fields

method: Method

Request method used for making requests.

github_client: Option<Octocrab>

GitHub client used for requests.

plugin_request_chain: RequestChain

The chain of plugins to be executed on each request.

max_retries: u64

Maximum number of retries per request before returning an error.

retry_wait_time: Duration

Initial wait time between retries of failed requests. This doubles after each failure.

accepted: HashSet<StatusCode>

Set of accepted return codes / status codes.

Unmatched return codes/ status codes are deemed as errors.

require_https: bool

Requires using HTTPS when it’s available.

This would treat unencrypted links as errors when HTTPS is available.

include_fragments: bool

Whether to check the existence of fragments in the response HTML files.

Will be disabled if the request method is HEAD.

fragment_checker: FragmentChecker

Utility for performing fragment checks in HTML files.

redirect_history: RedirectHistory

Keep track of HTTP redirections for reporting

host_pool: Arc<HostPool>

Optional host pool for per-host rate limiting.

When present, HTTP requests will be routed through this pool for rate limiting. When None, requests go directly through reqwest_client.

Implementations

impl WebsiteChecker

pub(crate) fn host_pool(&self) -> Arc<HostPool>

Get a reference to HostPool

pub(crate) fn new( method: Method, retry_wait_time: Duration, redirect_history: RedirectHistory, max_retries: u64, accepted: HashSet<StatusCode>, github_client: Option<Octocrab>, require_https: bool, plugin_request_chain: RequestChain, include_fragments: bool, host_pool: Arc<HostPool>, ) -> Self

pub(crate) async fn retry_request(&self, request: Request) -> Status

Retry requests up to max_retries times with an exponential backoff. Note that, in addition, there also is a host-specific backoff when host-specific rate limiting or errors are detected.

async fn check_default(&self, request: Request) -> Status

Check a URI using reqwest.

async fn check_html_fragment( &self, url: Url, status: Status, content: &str, file_type: FileType, ) -> Status

pub(crate) async fn check_website( &self, uri: &Uri, credentials: Option<BasicAuthCredentials>, ) -> Result<Status, ErrorKind>

Checks the given URI of a website.

Errors

This returns an Err if

• The URI is invalid.
• The request failed.
• The response status code is not accepted.
• The URI cannot be converted to HTTPS.

async fn handle_insecure_url( &self, uri: &Uri, default_chain: &Chain<Request, Status>, status: Status, ) -> Result<Status, ErrorKind>

Mark HTTP URLs as insecure, if the user required HTTPS and the URL is available under HTTPS.

async fn check_website_inner( &self, uri: &Uri, default_chain: &RequestChain, ) -> Status

Checks the given URI of a website.

Unsupported schemes will be ignored

Note: we use inner to improve compile times by avoiding monomorphization

Errors

This returns an Err if

• The URI is invalid.
• The request failed.
• The response status code is not accepted.

async fn handle_github(&self, status: Status, uri: &Uri) -> Status

async fn check_github(&self, uri: GithubUri) -> Status

Check a uri hosted on GitHub via the GitHub API.

Caveats

Files inside private repositories won’t get checked and instead would be reported as valid if the repository itself is reachable through the API.

A better approach would be to download the file through the API or clone the repo, but we chose the pragmatic approach.

Trait Implementations

impl Clone for WebsiteChecker

fn clone(&self) -> WebsiteChecker

Returns a duplicate of the value.

fn clone_from(&mut self, source: &Self)

Performs copy-assignment from source.

impl Debug for WebsiteChecker

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl Handler<Request, Status> for WebsiteChecker

fn handle<'life0, 'async_trait>( &'life0 mut self, input: Request, ) -> Pin<Box<dyn Future<Output = ChainResult<Request, Status>> + Send + 'async_trait>>

where Self: 'async_trait, 'life0: 'async_trait,

Given an input request, return a ChainResult to continue or stop the chain.