feat: add display UI with splash screen, dashboard, and RSSI sparkline
Implement full SSD1306 OLED display support: - Splash screen on boot (1.5s) with project name, version, board - Dashboard showing radio state, config, counters, RSSI/SNR - RSSI history sparkline (64-sample ring buffer, 2px-wide bars) - DisplayOn/DisplayOff host commands routed via dedicated channel - Pure render functions against generic DrawTarget (no hardware coupling) - Display on/off control bypasses radio task entirely Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
17624 files changed
+4564928
-38
No files matched your search
+10
@@ -0,0 +1,10 @@
|
||||
//! # Arithmetic
|
||||
//!
|
||||
//! This parses arithmetic expressions and directly evaluates them.
|
||||
//!
|
||||
//! See also [C-style Expressions][crate::_topic::language#c-style-expressions],
|
||||
//! for a more complex example.
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/arithmetic/parser.rs")]
|
||||
//! ```
|
||||
+44
@@ -0,0 +1,44 @@
|
||||
//! # Custom Errors
|
||||
//!
|
||||
//! A lot can be accomplished with the built-in error tools, like:
|
||||
//! - [`ContextError`]
|
||||
//! - [`Parser::context`]
|
||||
//! - [`cut_err`]
|
||||
//!
|
||||
//! *(see [tutorial][chapter_7])*
|
||||
//!
|
||||
//! Most other needs can likely be met by using a custom context type with [`ContextError`] instead
|
||||
//! of [`StrContext`].
|
||||
//! This will require implementing a custom renderer.
|
||||
//!
|
||||
//! ## `ParserError` Trait
|
||||
//!
|
||||
//! When needed, you can also create your own type that implements [`ParserError`].
|
||||
//!
|
||||
//! Optional traits include:
|
||||
//! - [`AddContext`]
|
||||
//! - [`FromExternalError`]
|
||||
//! - [`ErrorConvert`]
|
||||
//!
|
||||
//! There are multiple strategies for implementing support for [`AddContext`] and [`FromExternalError`]:
|
||||
//! - Make your error type generic over the context or external error
|
||||
//! - Require a trait for the context or external error and `Box` it
|
||||
//! - Make the context an enum like [`StrContext`]
|
||||
//! - Implement the trait multiple times, one for each concrete context or external error type,
|
||||
//! allowing custom behavior per type
|
||||
//!
|
||||
//! Example:
|
||||
//!```rust
|
||||
#![doc = include_str!("../../examples/custom_error.rs")]
|
||||
//!```
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::combinator::cut_err;
|
||||
use crate::error::ContextError;
|
||||
use crate::error::ErrorConvert;
|
||||
use crate::error::StrContext;
|
||||
use crate::Parser;
|
||||
use crate::_tutorial::chapter_7;
|
||||
use crate::error::AddContext;
|
||||
use crate::error::FromExternalError;
|
||||
use crate::error::ParserError;
|
||||
+8
@@ -0,0 +1,8 @@
|
||||
//! # Implementing `FromStr`
|
||||
//!
|
||||
//! The [`FromStr` trait][std::str::FromStr] provides
|
||||
//! a common interface to parse from a string.
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/css/parser.rs")]
|
||||
//! ```
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
//! # HTTP
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/http/parser.rs")]
|
||||
//! ```
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
//! # INI
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/ini/parser.rs")]
|
||||
//! ```
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
//! # json
|
||||
//!
|
||||
//! ```rust,ignore
|
||||
#![doc = include_str!("../../examples/json/parser_dispatch.rs")]
|
||||
//! ```
|
||||
+341
@@ -0,0 +1,341 @@
|
||||
//! # Elements of Programming Languages
|
||||
//!
|
||||
//! These are short recipes for accomplishing common tasks.
|
||||
//!
|
||||
//! * [Whitespace](#whitespace)
|
||||
//! + [Wrapper combinators that eat whitespace before and after a parser](#wrapper-combinators-that-eat-whitespace-before-and-after-a-parser)
|
||||
//! * [Comments](#comments)
|
||||
//! + [`// C++/EOL-style comments`](#-ceol-style-comments)
|
||||
//! + [`/* C-style comments */`](#-c-style-comments-)
|
||||
//! * [Identifiers](#identifiers)
|
||||
//! + [`Rust-Style Identifiers`](#rust-style-identifiers)
|
||||
//! * [Literal Values](#literal-values)
|
||||
//! + [Escaped Strings](#escaped-strings)
|
||||
//! + [Integers](#integers)
|
||||
//! - [Hexadecimal](#hexadecimal)
|
||||
//! - [Octal](#octal)
|
||||
//! - [Binary](#binary)
|
||||
//! - [Decimal](#decimal)
|
||||
//! + [Floating Point Numbers](#floating-point-numbers)
|
||||
//! * [C-style Expressions](#c-style-expressions)
|
||||
//!
|
||||
//! ## Whitespace
|
||||
//!
|
||||
//!
|
||||
//!
|
||||
//! ### Wrapper combinators that eat whitespace before and after a parser
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! error::ParserError,
|
||||
//! combinator::delimited,
|
||||
//! ascii::multispace0,
|
||||
//! };
|
||||
//!
|
||||
//! /// A combinator that takes a parser `inner` and produces a parser that also consumes both leading and
|
||||
//! /// trailing whitespace, returning the output of `inner`.
|
||||
//! fn ws<'a, F, O, E: ParserError<&'a str>>(inner: F) -> impl Parser<&'a str, O, E>
|
||||
//! where
|
||||
//! F: Parser<&'a str, O, E>,
|
||||
//! {
|
||||
//! delimited(
|
||||
//! multispace0,
|
||||
//! inner,
|
||||
//! multispace0
|
||||
//! )
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! To eat only trailing whitespace, replace `delimited(...)` with `terminated(&inner, multispace0)`.
|
||||
//! Likewise, the eat only leading whitespace, replace `delimited(...)` with `preceded(multispace0,
|
||||
//! &inner)`. You can use your own parser instead of `multispace0` if you want to skip a different set
|
||||
//! of lexemes.
|
||||
//!
|
||||
//! ## Comments
|
||||
//!
|
||||
//! ### `// C++/EOL-style comments`
|
||||
//!
|
||||
//! This version uses `%` to start a comment, does not consume the newline character, and returns an
|
||||
//! output of `()`.
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! error::ParserError,
|
||||
//! token::take_till,
|
||||
//! };
|
||||
//!
|
||||
//! pub fn peol_comment<'a, E: ParserError<&'a str>>(i: &mut &'a str) -> ModalResult<(), E>
|
||||
//! {
|
||||
//! ('%', take_till(1.., ['\n', '\r']))
|
||||
//! .void() // Output is thrown away.
|
||||
//! .parse_next(i)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! ### `/* C-style comments */`
|
||||
//!
|
||||
//! Inline comments surrounded with sentinel literals `(*` and `*)`. This version returns an output of `()`
|
||||
//! and does not handle nested comments.
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! error::ParserError,
|
||||
//! token::take_until,
|
||||
//! };
|
||||
//!
|
||||
//! pub fn pinline_comment<'a, E: ParserError<&'a str>>(i: &mut &'a str) -> ModalResult<(), E> {
|
||||
//! (
|
||||
//! "(*",
|
||||
//! take_until(0.., "*)"),
|
||||
//! "*)"
|
||||
//! )
|
||||
//! .void() // Output is thrown away.
|
||||
//! .parse_next(i)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! ## Identifiers
|
||||
//!
|
||||
//! ### `Rust-Style Identifiers`
|
||||
//!
|
||||
//! Parsing identifiers that may start with a letter (or underscore) and may contain underscores,
|
||||
//! letters and numbers may be parsed like this:
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! stream::AsChar,
|
||||
//! token::take_while,
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! pub fn identifier<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! (
|
||||
//! one_of(|c: char| c.is_alpha() || c == '_'),
|
||||
//! take_while(0.., |c: char| c.is_alphanum() || c == '_')
|
||||
//! )
|
||||
//! .take()
|
||||
//! .parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Let's say we apply this to the identifier `hello_world123abc`. The first element of the tuple
|
||||
//! would uses [`one_of`][crate::token::one_of] which would take `h`. The tuple ensures that
|
||||
//! `ello_world123abc` will be piped to the next [`take_while`][crate::token::take_while] parser,
|
||||
//! which takes every remaining character. However, the tuple returns a tuple of the results
|
||||
//! of its sub-parsers. The [`take`][crate::Parser::take] parser produces a `&str` of the
|
||||
//! input text that was parsed, which in this case is the entire `&str` `hello_world123abc`.
|
||||
//!
|
||||
//! ## Literal Values
|
||||
//!
|
||||
//! ### Escaped Strings
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/string/parser.rs")]
|
||||
//! ```
|
||||
//!
|
||||
//! See also [`take_escaped`] and [`escaped`].
|
||||
//!
|
||||
//! ### Integers
|
||||
//!
|
||||
//! The following recipes all return string slices rather than integer values. How to obtain an
|
||||
//! integer value instead is demonstrated for hexadecimal integers. The others are similar.
|
||||
//!
|
||||
//! The parsers allow the grouping character `_`, which allows one to group the digits by byte, for
|
||||
//! example: `0xA4_3F_11_28`. If you prefer to exclude the `_` character, the lambda to convert from a
|
||||
//! string slice to an integer value is slightly simpler. You can also strip the `_` from the string
|
||||
//! slice that is returned, which is demonstrated in the second hexadecimal number parser.
|
||||
//!
|
||||
//! #### Hexadecimal
|
||||
//!
|
||||
//! The parser outputs the string slice of the digits without the leading `0x`/`0X`.
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! combinator::alt,
|
||||
//! combinator::repeat,
|
||||
//! combinator::{preceded, terminated},
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! fn hexadecimal<'s>(input: &mut &'s str) -> ModalResult<&'s str> { // <'a, E: ParserError<&'a str>>
|
||||
//! preceded(
|
||||
//! alt(("0x", "0X")),
|
||||
//! repeat(1..,
|
||||
//! terminated(one_of(('0'..='9', 'a'..='f', 'A'..='F')), repeat(0.., '_').map(|()| ()))
|
||||
//! ).map(|()| ()).take()
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! If you want it to return the integer value instead, use map:
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! combinator::alt,
|
||||
//! combinator::repeat,
|
||||
//! combinator::{preceded, terminated},
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! fn hexadecimal_value(input: &mut &str) -> ModalResult<i64> {
|
||||
//! preceded(
|
||||
//! alt(("0x", "0X")),
|
||||
//! repeat(1..,
|
||||
//! terminated(one_of(('0'..='9', 'a'..='f', 'A'..='F')), repeat(0.., '_').map(|()| ()))
|
||||
//! ).map(|()| ()).take()
|
||||
//! ).try_map(
|
||||
//! |out: &str| i64::from_str_radix(&str::replace(&out, "_", ""), 16)
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! See also [`hex_uint`]
|
||||
//!
|
||||
//! #### Octal
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! combinator::alt,
|
||||
//! combinator::repeat,
|
||||
//! combinator::{preceded, terminated},
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! fn octal<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! preceded(
|
||||
//! alt(("0o", "0O")),
|
||||
//! repeat(1..,
|
||||
//! terminated(one_of('0'..='7'), repeat(0.., '_').map(|()| ()))
|
||||
//! ).map(|()| ()).take()
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! #### Binary
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! combinator::alt,
|
||||
//! combinator::repeat,
|
||||
//! combinator::{preceded, terminated},
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! fn binary<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! preceded(
|
||||
//! alt(("0b", "0B")),
|
||||
//! repeat(1..,
|
||||
//! terminated(one_of('0'..='1'), repeat(0.., '_').map(|()| ()))
|
||||
//! ).map(|()| ()).take()
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! #### Decimal
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! combinator::repeat,
|
||||
//! combinator::terminated,
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! fn decimal<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! repeat(1..,
|
||||
//! terminated(one_of('0'..='9'), repeat(0.., '_').map(|()| ()))
|
||||
//! ).map(|()| ())
|
||||
//! .take()
|
||||
//! .parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! See also [`dec_uint`] and [`dec_int`]
|
||||
//!
|
||||
//! ### Floating Point Numbers
|
||||
//!
|
||||
//! The following is adapted from [the Python parser by Valentin Lorentz](https://github.com/ProgVal/rust-python-parser/blob/master/src/numbers.rs).
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! use winnow::{
|
||||
//! combinator::alt,
|
||||
//! combinator::repeat,
|
||||
//! combinator::opt,
|
||||
//! combinator::{preceded, terminated},
|
||||
//! token::one_of,
|
||||
//! };
|
||||
//!
|
||||
//! fn float<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! alt((
|
||||
//! // Case one: .42
|
||||
//! (
|
||||
//! '.',
|
||||
//! decimal,
|
||||
//! opt((
|
||||
//! one_of(['e', 'E']),
|
||||
//! opt(one_of(['+', '-'])),
|
||||
//! decimal
|
||||
//! ))
|
||||
//! ).take()
|
||||
//! , // Case two: 42e42 and 42.42e42
|
||||
//! (
|
||||
//! decimal,
|
||||
//! opt(preceded(
|
||||
//! '.',
|
||||
//! decimal,
|
||||
//! )),
|
||||
//! one_of(['e', 'E']),
|
||||
//! opt(one_of(['+', '-'])),
|
||||
//! decimal
|
||||
//! ).take()
|
||||
//! , // Case three: 42. and 42.42
|
||||
//! (
|
||||
//! decimal,
|
||||
//! '.',
|
||||
//! opt(decimal)
|
||||
//! ).take()
|
||||
//! )).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn decimal<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! repeat(1..,
|
||||
//! terminated(one_of('0'..='9'), repeat(0.., '_').map(|()| ()))
|
||||
//! ).
|
||||
//! map(|()| ())
|
||||
//! .take()
|
||||
//! .parse_next(input)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! See also [`float`]
|
||||
//!
|
||||
//! ## C-style Expressions
|
||||
//!
|
||||
//! An example using the [`expression()`] parser to build an abstract syntax tree
|
||||
//! for C-style expressions.
|
||||
//!
|
||||
//! The operator precedence level is based on the [C language](c-precedence).
|
||||
//!
|
||||
//! [c-precedence]: https://en.cppreference.com/w/c/language/operator_precedence.html
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/c_expression/parser.rs")]
|
||||
//! ```
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::ascii::dec_int;
|
||||
use crate::ascii::dec_uint;
|
||||
use crate::ascii::escaped;
|
||||
use crate::ascii::float;
|
||||
use crate::ascii::hex_uint;
|
||||
use crate::ascii::take_escaped;
|
||||
use crate::combinator::expression;
|
||||
+23
@@ -0,0 +1,23 @@
|
||||
//! # Lexing and Parsing
|
||||
//!
|
||||
//! ## Parse to AST
|
||||
//!
|
||||
//! The simplest way to write a parser is to parse directly to the AST.
|
||||
//!
|
||||
//! Example:
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/arithmetic/parser_ast.rs")]
|
||||
//! ```
|
||||
//!
|
||||
//! ## Lexing
|
||||
//!
|
||||
//! However, there are times when you may want to separate lexing from parsing.
|
||||
//! Winnow provides [`TokenSlice`] to simplify this.
|
||||
//!
|
||||
//! Example:
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/arithmetic/parser_lexer.rs")]
|
||||
//! ```
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::stream::TokenSlice;
|
||||
+43
@@ -0,0 +1,43 @@
|
||||
//! # Special Topics
|
||||
//!
|
||||
//! These are short recipes for accomplishing common tasks.
|
||||
//!
|
||||
//! - [Why `winnow`?][why]
|
||||
//! - [For `nom` users][nom]
|
||||
//! - Formats:
|
||||
//! - [Elements of Programming Languages][language]
|
||||
//! - [Arithmetic][arithmetic]
|
||||
//! - [s-expression][s_expression]
|
||||
//! - [json]
|
||||
//! - [INI][ini]
|
||||
//! - [HTTP][http]
|
||||
//! - Special Topics:
|
||||
//! - [Implementing `FromStr`][fromstr]
|
||||
//! - [Performance][performance]
|
||||
//! - [Parsing Partial Input][partial]
|
||||
//! - [Lexing and Parsing][lexing]
|
||||
//! - [Custom stream or token][stream]
|
||||
//! - [Custom errors][error]
|
||||
//! - [Debugging][crate::_tutorial::chapter_8]
|
||||
//!
|
||||
//! See also parsers written with `winnow`:
|
||||
//!
|
||||
//! - [`toml_edit`](https://crates.io/crates/toml_edit)
|
||||
//! - [`hcl-edit`](https://crates.io/crates/hcl-edit)
|
||||
#![allow(clippy::std_instead_of_core)]
|
||||
#![allow(clippy::test_attr_in_doctest)]
|
||||
|
||||
pub mod arithmetic;
|
||||
pub mod error;
|
||||
pub mod fromstr;
|
||||
pub mod http;
|
||||
pub mod ini;
|
||||
pub mod json;
|
||||
pub mod language;
|
||||
pub mod lexing;
|
||||
pub mod nom;
|
||||
pub mod partial;
|
||||
pub mod performance;
|
||||
pub mod s_expression;
|
||||
pub mod stream;
|
||||
pub mod why;
|
||||
+240
@@ -0,0 +1,240 @@
|
||||
//! # For `nom` users
|
||||
//!
|
||||
//! ## Migrating from `nom`
|
||||
//!
|
||||
//! For comparisons with `nom`, see
|
||||
//! - [Why `winnow`][super::why]
|
||||
//! - [parse-rosetta-rs](https://github.com/rosetta-rs/parse-rosetta-rs/)
|
||||
//!
|
||||
//! What approach you take depends on the size and complexity of your parser.
|
||||
//! For small, simple parsers, its likely easiest to directly port from `nom`.
|
||||
//! When trying to look for the equivalent of a `nom` combinator, search in the docs for the name
|
||||
//! of the `nom` combinator. It is expected that, where names diverge, a doc alias exists.
|
||||
//! See also the [List of combinators][crate::combinator].
|
||||
//!
|
||||
//! ### Complex migrations
|
||||
//!
|
||||
//! For larger parsers, it is likely best to take smaller steps
|
||||
//! - Easier to debug when something goes wrong
|
||||
//! - Deprecation messages will help assist through the process
|
||||
//!
|
||||
//! The workflow goes something like:
|
||||
//! 1. Run `cargo rm nom && cargo add winnow@0.3`
|
||||
//! 1. Ensure everything compiles and tests pass, ignoring deprecation messages (see [migration
|
||||
//! notes](https://github.com/winnow-rs/winnow/blob/main/CHANGELOG.md#nom-migration-guide))
|
||||
//! 1. Commit
|
||||
//! 1. Switch any `impl FnMut(I) -> IResult<I, O, E>` to `impl Parser<I, O, E>`
|
||||
//! 1. Resolve deprecation messages
|
||||
//! 1. Commit
|
||||
//! 1. Run `cargo add winnow@0.4`
|
||||
//! 1. Ensure everything compiles and tests pass, ignoring deprecation messages (see [changelog](https://github.com/winnow-rs/winnow/blob/main/CHANGELOG.md#040---2023-03-18) for more details)
|
||||
//! 1. Commit
|
||||
//! 1. Resolve deprecation messages
|
||||
//! 1. Commit
|
||||
//! 1. Run `cargo add winnow@0.5`
|
||||
//! 1. Ensure everything compiles and tests pass, ignoring deprecation messages
|
||||
//! (see [migration notes](https://github.com/winnow-rs/winnow/blob/main/CHANGELOG.md#050---2023-07-13))
|
||||
//! 1. Commit
|
||||
//! 1. Resolve deprecation messages
|
||||
//! 1. Commit
|
||||
//!
|
||||
//! ### Examples
|
||||
//!
|
||||
//! For example migrations, see
|
||||
//! - [git-config-env](https://github.com/gitext-rs/git-config-env/pull/11) (nom to winnow 0.3)
|
||||
//! - [git-conventional](https://github.com/crate-ci/git-conventional/pull/37) (nom to winnow 0.3,
|
||||
//! adds explicit tracing for easier debugging)
|
||||
//! - [typos](https://github.com/crate-ci/typos/pull/664) (nom to winnow 0.3)
|
||||
//! - [cargo-smart-release](https://github.com/Byron/gitoxide/pull/948) (gradual migration from nom
|
||||
//! to winnow 0.5)
|
||||
//! - [gix-config](https://github.com/Byron/gitoxide/pull/951) (gradual migration from nom
|
||||
//! to winnow 0.5)
|
||||
//! - [gix-protocol](https://github.com/Byron/gitoxide/pull/1009) (gradual migration from nom
|
||||
//! to winnow 0.5)
|
||||
//! - [gitoxide](https://github.com/Byron/gitoxide/pull/956) (gradual migration from nom
|
||||
//! to winnow 0.5)
|
||||
//!
|
||||
//! ## Differences
|
||||
//!
|
||||
//! These are key differences to help Nom users adapt to writing parsers with Winnow.
|
||||
//!
|
||||
//! ### Renamed APIs
|
||||
//!
|
||||
//! Names have changed for consistency or clarity.
|
||||
//!
|
||||
//! To find a parser you are looking for,
|
||||
//! - Search the docs for the `nom` parser
|
||||
//! - See the [List of combinators][crate::combinator]
|
||||
//!
|
||||
//! ### GATs
|
||||
//!
|
||||
//! `nom` v8 back-propagates how you will use a parser to parser functions using a language feature
|
||||
//! called GATs.
|
||||
//! Winnow has made the conscious choice not to use this feature, finding alternative ways of
|
||||
//! getting most of the benefits.
|
||||
//!
|
||||
//! Benefits of GATs:
|
||||
//! - Performance as the compiler is able to instantiate copies of a parser that are
|
||||
//! better tailored to how it will be used, like discarding unused allocations for output or
|
||||
//! errors.
|
||||
//!
|
||||
//! Benefits of not using GATs:
|
||||
//! - Predictable performance:
|
||||
//! With GATs, seemingly innocuous changes like choosing to hand write a parser using idiomatic function parsers
|
||||
//! (`fn(&mut I) -> Result<O>`) can cause surprising slow downs because these functions sever the back-propagation from GATs.
|
||||
//! The causes of these slowdowns could be hard to identify by inspection or profiling.
|
||||
//! - No "eek out X% perf improvement" pressure to contort a parser
|
||||
//! that is more easily written imperatively
|
||||
//! to be written declaratively
|
||||
//! so it can preserve the back-propagation from GATs.
|
||||
//! - Built-in parsers serve are can serve as examples to users of idiomatic function parsers
|
||||
//! (`fn(&mut I) -> Result<O>`).
|
||||
//! With GATs, built-in parsers tend to be complex implementations of traits.
|
||||
//! - Faster build times and smaller binary size as parsers only need to be generated for one mode,
|
||||
//! not upto 8
|
||||
//!
|
||||
//! #### Partial/streaming parsers
|
||||
//!
|
||||
//! `nom` v8 back-propagates whether `Parser::parse_complete` was used to select `complete`
|
||||
//! parsers.
|
||||
//! Previously, users had ensure consistently using a parser from the `streaming` or `complete` module.
|
||||
//!
|
||||
//! Instead, you tag the input type (`I`) by wrapping it in [`Partial<I>`] and parsers will adjust
|
||||
//! their behavior accordingly.
|
||||
//! See [partial] special topic.
|
||||
//!
|
||||
//! #### Eliding Output
|
||||
//!
|
||||
//! `nom` v8 back-propagates whether an Output will be used and skips its creation.
|
||||
//! For example, `value(Null, many0(_))` will avoid creating and pushing to a `Vec`.
|
||||
//! Previously, users had to select `count_many0` over `many0` to avoid creating a `Vec`.
|
||||
//!
|
||||
//! Instead, `repeat` returns an `impl Accumulate<T>` which could be a `Vec`, a `usize` for `count`
|
||||
//! variants, or `()` to do no extra work.
|
||||
//!
|
||||
//! #### Eliding Backtracked Errors
|
||||
//!
|
||||
//! Under the hood, [`alt`] is an `if-not-error-else` ladder, see [`_tutorial::chapter_3`].
|
||||
//! nom v8 back-propagates whether the error will be discarded and avoids any expensive work done
|
||||
//! for rich error messages.
|
||||
//!
|
||||
//! Instead, [`ContextError`] and other changes have made it so errors have very little overhead.
|
||||
//! [`dispatch!`] can also be used in some situations to avoid `alt`s overhead.
|
||||
//!
|
||||
//! ### Parsers return [`Stream::Slice`], rather than [`Stream`]
|
||||
//!
|
||||
//! In `nom`, parsers like [`take_while`] parse a [`Stream`] and return a [`Stream`].
|
||||
//! When wrapping the input, like with [`Stateful`],
|
||||
//! you have to unwrap the input to integrate it in your application,
|
||||
//! and it requires [`Stream`] to be `Clone`
|
||||
//! (which requires `RefCell` for mutable external state and can be expensive).
|
||||
//!
|
||||
//! Instead, [`Stream::Slice`] was added to track the intended type for parsers to return.
|
||||
//! If you want to then parse the slice, you then need to take it and turn it back into a
|
||||
//! [`Stream`].
|
||||
//!
|
||||
//! ### `&mut I`
|
||||
//!
|
||||
//! `winnow` switched from pure-function parser (`Fn(I) -> (I, O)` to `Fn(&mut I) -> O`).
|
||||
//! On error, `i` is left pointing at where the error happened.
|
||||
//!
|
||||
//! Benefits of `Fn(&mut I) -> O`:
|
||||
//! - Cleaner code: Removes need to pass `i` everywhere and makes changes to `i` more explicit
|
||||
//! - Correctness: No forgetting to chain `i` through a parser
|
||||
//! - Flexibility: `I` does not need to be `Copy` or even `Clone`. For example, [`Stateful`] can use `&mut S` instead of `RefCell<S>`.
|
||||
//! - Performance: `Result::Ok` is smaller without `i`, reducing the risk that the output will be
|
||||
//! returned on the stack, rather than the much faster CPU registers.
|
||||
//! `Result::Err` can also be smaller because the error type does not need to carry `i` to point
|
||||
//! to the error.
|
||||
//! See also [#72](https://github.com/winnow-rs/winnow/issues/72).
|
||||
//!
|
||||
//! Benefits of `Fn(I) -> (I, O)`:
|
||||
//! - Pure functions can be easier to reason about
|
||||
//! - Less boilerplate in some situations (see below)
|
||||
//! - Less syntactic noise in some situations (see below)
|
||||
//!
|
||||
//! When returning a slice from the input, you have to add a lifetime:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! fn foo<'i>(i: &mut &'i str) -> ModalResult<&'i str> {
|
||||
//! # Ok("")
|
||||
//! // ...
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! When writing a closure, you need to annotate the type
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::error::ContextError;
|
||||
//! # let mut input = "";
|
||||
//! # fn foo<'i>() -> impl ModalParser<&'i str, &'i str, ContextError> {
|
||||
//! alt((
|
||||
//! |i: &mut _| {
|
||||
//! # Ok("")
|
||||
//! // ...
|
||||
//! },
|
||||
//! |i: &mut _| {
|
||||
//! # Ok("")
|
||||
//! // ...
|
||||
//! },
|
||||
//! ))
|
||||
//! # }
|
||||
//! ```
|
||||
//! *(at least the full type isn't needed)*
|
||||
//!
|
||||
//! To save and restore from intermediate states, [`Stream::checkpoint`] and [`Stream::reset`] can help:
|
||||
//! ```rust
|
||||
//! use winnow::prelude::*;
|
||||
//! # let mut i = "";
|
||||
//! # let i = &mut i;
|
||||
//!
|
||||
//! let start = i.checkpoint();
|
||||
//! // ...
|
||||
//! i.reset(&start);
|
||||
//! ```
|
||||
//!
|
||||
//! When the Output of a parser is a slice, you have to add a lifetime:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! fn foo<'i>(i: &mut &'i str) -> ModalResult<&'i str> {
|
||||
//! // ...
|
||||
//! # winnow::token::rest.parse_next(i)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! When writing a closure, you need to annotate the type:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::combinator::trace;
|
||||
//! fn foo(i: &mut &str) -> ModalResult<usize> {
|
||||
//! trace("foo", |i: &mut _| {
|
||||
//! // ...
|
||||
//! # Ok(0)
|
||||
//! }).parse_next(i)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! ### Optional [`ErrMode`]
|
||||
//!
|
||||
//! Called `Err` in `nom`, [`ErrMode`] is responsible for
|
||||
//! - Deciding whether to backtrack and try another branch in cases like `alt` or report back to
|
||||
//! the error back to users
|
||||
//! - Tracking incomplete input on partial parsing
|
||||
//!
|
||||
//! As this isn't needed in every parser, it was made optional. [`ModalResult`] is a convenience
|
||||
//! type for using [`ErrMode`].
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::_topic::partial;
|
||||
use crate::_tutorial;
|
||||
use crate::combinator::alt;
|
||||
use crate::combinator::dispatch;
|
||||
use crate::error::ContextError;
|
||||
use crate::error::ErrMode;
|
||||
use crate::error::ModalResult;
|
||||
use crate::stream::Accumulate;
|
||||
use crate::stream::Partial;
|
||||
use crate::stream::Stateful;
|
||||
use crate::stream::Stream;
|
||||
use crate::token::take_while;
|
||||
+46
@@ -0,0 +1,46 @@
|
||||
//! # Parsing Partial Input
|
||||
//!
|
||||
//! Typically, the input being parsed is all in-memory, or is complete. Some data sources are too
|
||||
//! large to fit into memory, only allowing parsing an incomplete or [`Partial`] subset of the
|
||||
//! data, requiring incrementally parsing.
|
||||
//!
|
||||
//! By wrapping a stream, like `&[u8]`, with [`Partial`], parsers will report when the data is
|
||||
//! [`Incomplete`] and more input is [`Needed`], allowing the caller to stream-in additional data
|
||||
//! to be parsed. The data is then parsed a chunk at a time.
|
||||
//!
|
||||
//! Chunks are typically defined by either:
|
||||
//! - A header reporting the number of bytes, like with [`length_and_then`]
|
||||
//! - [`Partial`] can explicitly be changed to being complete once the specified bytes are
|
||||
//! acquired via [`StreamIsPartial::complete`].
|
||||
//! - A delimiter, like with [ndjson](https://github.com/ndjson/ndjson-spec/)
|
||||
//! - You can parse up-to the delimiter or do a `take_until(0.., delim).and_then(parser)`
|
||||
//!
|
||||
//! If the chunks are not homogeneous, a state machine will be needed to track what the expected
|
||||
//! parser is for the next chunk.
|
||||
//!
|
||||
//! Caveats:
|
||||
//! - `winnow` takes the approach of re-parsing from scratch. Chunks should be relatively small to
|
||||
//! prevent the re-parsing overhead from dominating.
|
||||
//! - Parsers like [`repeat`] do not know when an `eof` is from insufficient data or the end of the
|
||||
//! stream, causing them to always report [`Incomplete`].
|
||||
//!
|
||||
//! # Example
|
||||
//!
|
||||
//! `main.rs`:
|
||||
//! ```rust,ignore
|
||||
#![doc = include_str!("../../examples/ndjson/main.rs")]
|
||||
//! ```
|
||||
//!
|
||||
//! `parser.rs`:
|
||||
//! ```rust,ignore
|
||||
#![doc = include_str!("../../examples/ndjson/parser.rs")]
|
||||
//! ```
|
||||
|
||||
#![allow(unused_imports)] // Used for intra-doc links
|
||||
|
||||
use crate::binary::length_and_then;
|
||||
use crate::combinator::repeat;
|
||||
use crate::error::ErrMode::Incomplete;
|
||||
use crate::error::Needed;
|
||||
use crate::stream::Partial;
|
||||
use crate::stream::StreamIsPartial;
|
||||
+55
@@ -0,0 +1,55 @@
|
||||
//! # Performance
|
||||
//!
|
||||
//! ## Runtime Performance
|
||||
//!
|
||||
//! See also the general Rust [Performance Book](https://nnethercote.github.io/perf-book/)
|
||||
//!
|
||||
//! Tips
|
||||
//! - Try `cargo add winnow -F simd`. For some it offers significant performance improvements
|
||||
//! - When enough cases of an [`alt`] have unique prefixes, prefer [`dispatch`]
|
||||
//! - When parsing text, try to parse as bytes (`u8`) rather than `char`s ([`BStr`] can make
|
||||
//! debugging easier)
|
||||
//! - Find simplified subsets of the grammar to parse, falling back to the full grammar when it
|
||||
//! doesn't work. For example, when parsing json strings, parse them without support for escapes,
|
||||
//! falling back to escape support if it fails.
|
||||
//! - Watch for large return types. A surprising place these can show up is when chaining parsers
|
||||
//! with a tuple.
|
||||
//!
|
||||
//! ## Build-time Performance
|
||||
//!
|
||||
//! Returning complex types as `impl Trait` can negatively impact build times. This can hit in
|
||||
//! surprising cases like:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! fn foo<I, O, E>() -> impl Parser<I, O, E>
|
||||
//! # where
|
||||
//! # I: winnow::stream::Stream<Token=O>,
|
||||
//! # I: winnow::stream::StreamIsPartial,
|
||||
//! # E: winnow::error::ParserError<I>,
|
||||
//! {
|
||||
//! // ...some chained combinators...
|
||||
//! # winnow::token::any
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Instead, wrap the combinators in a closure to simplify the type:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! fn foo<I, O, E>() -> impl Parser<I, O, E>
|
||||
//! # where
|
||||
//! # I: winnow::stream::Stream<Token=O>,
|
||||
//! # I: winnow::stream::StreamIsPartial,
|
||||
//! # E: winnow::error::ParserError<I>,
|
||||
//! {
|
||||
//! move |input: &mut I| {
|
||||
//! // ...some chained combinators...
|
||||
//! # winnow::token::any
|
||||
//! .parse_next(input)
|
||||
//! }
|
||||
//! }
|
||||
//! ```
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::combinator::alt;
|
||||
use crate::combinator::dispatch;
|
||||
use crate::stream::BStr;
|
||||
@@ -0,0 +1,5 @@
|
||||
//! # s-expression
|
||||
//!
|
||||
//! ```rust
|
||||
#![doc = include_str!("../../examples/s_expression/parser.rs")]
|
||||
//! ```
|
||||
+63
@@ -0,0 +1,63 @@
|
||||
//! # Custom [`Stream`]
|
||||
//!
|
||||
//! `winnow` is batteries included with support for
|
||||
//! - Basic inputs like `&str`, newtypes with
|
||||
//! - Improved debug output like [`Bytes`]
|
||||
//! - [`Stateful`] for passing state through your parser, like tracking recursion
|
||||
//! depth
|
||||
//! - [`LocatingSlice`] for looking up the absolute position of a token
|
||||
//!
|
||||
//! ## Implementing a custom token
|
||||
//!
|
||||
//! The first level of customization is parsing [`&[MyItem]`][Stream#impl-Stream-for-%26%5BT%5D]
|
||||
//! or [`TokenSlice<MyItem>`].
|
||||
//!
|
||||
//! The basic traits you may want for a custom token type are:
|
||||
//!
|
||||
//! | trait | usage |
|
||||
//! |---|---|
|
||||
//! | [`AsChar`] |Transforms common types to a char for basic token parsing|
|
||||
//! | [`ContainsToken`] |Look for the token in the given set|
|
||||
//!
|
||||
//! See also [`TokenSlice<MyItem>`], [lexing].
|
||||
//!
|
||||
//! ## Implementing a custom stream
|
||||
//!
|
||||
//! Let's assume we have an input type we'll call `MyStream`.
|
||||
//! `MyStream` is a sequence of `MyItem` tokens.
|
||||
//!
|
||||
//! The goal is to define parsers with this signature: `&mut MyStream -> ModalResult<Output>`.
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # type MyStream<'i> = &'i str;
|
||||
//! # type Output<'i> = &'i str;
|
||||
//! fn parser<'s>(i: &mut MyStream<'s>) -> ModalResult<Output<'s>> {
|
||||
//! "test".parse_next(i)
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Like above, you'll need to implement the related token traits for `MyItem`.
|
||||
//!
|
||||
//! The traits you may want to implement for `MyStream` include:
|
||||
//!
|
||||
//! | trait | usage |
|
||||
//! |---|---|
|
||||
//! | [`Stream`] |Core trait for driving parsing|
|
||||
//! | [`StreamIsPartial`] | Marks the input as being the complete buffer or a partial buffer for streaming input |
|
||||
//! | [`AsBytes`] |Casts the input type to a byte slice|
|
||||
//! | [`AsBStr`] |Casts the input type to a slice of ASCII / UTF-8-like bytes|
|
||||
//! | [`Compare`] |Character comparison operations|
|
||||
//! | [`FindSlice`] |Look for a substring in self|
|
||||
//! | [`Location`] |Calculate location within initial input|
|
||||
//! | [`Offset`] |Calculate the offset between slices|
|
||||
//!
|
||||
//! And for `&[MyItem]` (slices returned by [`Stream`]):
|
||||
//!
|
||||
//! | trait | usage |
|
||||
//! |---|---|
|
||||
//! | [`SliceLen`] |Calculate the input length|
|
||||
//! | [`ParseSlice`] |Used to integrate `&str`'s `parse()` method|
|
||||
|
||||
#![allow(unused_imports)] // Here for intra-dock links
|
||||
use super::lexing;
|
||||
use crate::stream::*;
|
||||
+110
@@ -0,0 +1,110 @@
|
||||
//! # Why `winnow`?
|
||||
//!
|
||||
//! To answer this question, it will be useful to contrast this with other approaches to parsing.
|
||||
//!
|
||||
//! <div class="warning">
|
||||
//!
|
||||
//! **Note:** This will focus on principles and priorities. For a deeper and wider
|
||||
//! comparison with other Rust parser libraries, see
|
||||
//! [parse-rosetta-rs](https://github.com/rosetta-rs/parse-rosetta-rs).
|
||||
//!
|
||||
//! </div>
|
||||
//!
|
||||
//! ## Hand-written parsers
|
||||
//!
|
||||
//! Typically, a hand-written parser gives you the flexibility to get
|
||||
//! - Fast parse performance
|
||||
//! - Fast compile-time
|
||||
//! - Small binary sizes
|
||||
//! - High quality error message
|
||||
//! - Fewer dependencies to audit
|
||||
//!
|
||||
//! However, this comes at the cost of doing it all yourself, including
|
||||
//! - Optimizing for each of the above characteristics you care about
|
||||
//! - Ensuring the safety of any `unsafe` code (buffer overflows being a common bug with parsers)
|
||||
//! - Being aware of, familiar with, and correctly implement the relevant algorithms.
|
||||
//! matklad, who has written two rust compile frontends, commented
|
||||
//! ["I’ve implemented a production-grade Pratt parser once, but I no longer immediately understand that code :-)"](https://matklad.github.io/2020/04/13/simple-but-powerful-pratt-parsing.html)
|
||||
//!
|
||||
//! This approach works well if:
|
||||
//! - Your format is small and is unlikely to change
|
||||
//! - Your format is large but you have people who can focus solely on parsing, like with large
|
||||
//! programming languages
|
||||
//!
|
||||
//! ## `winnow`
|
||||
//!
|
||||
//! Unlike traditional programming language parsers that use
|
||||
//! [lex](https://en.wikipedia.org/wiki/Lex_(software)) or
|
||||
//! [yacc](https://en.wikipedia.org/wiki/Yacc), you can think of `winnow` as a general version of
|
||||
//! the helpers you would create along the way to writing a hand-written parser.
|
||||
//!
|
||||
//! `winnow` includes support for:
|
||||
//! - Zero-copy parsing
|
||||
//! - [Parse traces][trace] for easier debugging
|
||||
//! - [Streaming parsing][Partial] for network communication or large file
|
||||
//! - [Stateful] parsers
|
||||
//!
|
||||
//! For binary formats, `winnow` includes:
|
||||
//! - [A hexadecimal view][crate::Bytes] in [trace]
|
||||
//! - [TLV](https://en.wikipedia.org/wiki/Type-length-value) (e.g. [`length_take`])
|
||||
//! - Some common parsers to help get started, like numbers
|
||||
//!
|
||||
//! For text formats, `winnow` includes:
|
||||
//! - [Tracking of spans][crate::LocatingSlice]
|
||||
//! - [A textual view when parsing as bytes][crate::BStr] in [trace]
|
||||
//! - Ability to evaluate directly, parse to an AST, or lex and parse the format
|
||||
//!
|
||||
//! This works well for:
|
||||
//! - Prototyping for what will be a hand-written parser
|
||||
//! - When you want to minimize the work to evolve your format
|
||||
//! - When you don't have contributors focused solely on parsing and your grammar is large enough
|
||||
//! to be unwieldy to hand write.
|
||||
//!
|
||||
//! ## `nom`
|
||||
//!
|
||||
//! `winnow` is a fork of the venerable [`nom`](https://crates.io/crates/nom). The difference
|
||||
//! between them is largely in priorities. `nom` prioritizes:
|
||||
//! - Lower churn for existing users while `winnow` is trying to find ways to make things better
|
||||
//! for the parsers yet to be written.
|
||||
//! - Having a small core, relying on external crates like
|
||||
//! [`nom-locate`](https://crates.io/crates/nom_locate) and
|
||||
//! [`nom-supreme`](https://crates.io/crates/nom-supreme), encouraging flexibility among users
|
||||
//! and to not block users on new features being merged while `winnow` aims to include all the
|
||||
//! fundamentals for parsing to ensure the experience is cohesive and high quality.
|
||||
//!
|
||||
//! For more details, see the [design differences][super::nom#api-differences].
|
||||
//!
|
||||
//! See also our [nom migration guide][super::nom#migrating-from-nom].
|
||||
//!
|
||||
//! ## `chumsky`
|
||||
//!
|
||||
//! [`chumsky`](https://crates.io/crates/chumsky) is an up and coming parser-combinator library
|
||||
//! that includes advanced features like error recovery.
|
||||
//!
|
||||
//! Probably the biggest diverging philosophy is `chumsky`s stance:
|
||||
//!
|
||||
//! > "If you need to implement either `Parser` or `Strategy` by hand, that's a problem that needs fixing".
|
||||
//!
|
||||
//! This is under "batteries included" but it also ties into the feeling that `chumsky` acts more like
|
||||
//! a framework. Instead of composing together helpers, you are expected to do everything through
|
||||
//! their system to the point that it is non-trivial to implement their `Parser` trait and are
|
||||
//! encouraged to use the
|
||||
//! [`custom`](https://docs.rs/chumsky/0.9.0/chumsky/primitive/fn.custom.html) helper. This
|
||||
//! requires re-framing everything to fit within their model and makes the code harder to understand
|
||||
//! and debug as you are working with abstract operations that will eventually be applied
|
||||
//! rather than directly with the parsers.
|
||||
//!
|
||||
//! In contrast, `winnow` is an introspectable toolbox that can easily be customized at any level.
|
||||
//! Probably the biggest thing that `winnow` loses out on is optimizations from ["parse modes" via
|
||||
//! GATs](https://github.com/zesterer/chumsky/pull/82) which allows downstream parsers to tell
|
||||
//! upstream parsers when information will be discarded, allowing bypassing expensive operations,
|
||||
//! like allocations. This requires a lot more complex interaction with parsers that isn't as
|
||||
//! trivial to do with bare functions which would lose out on any of that side-band information.
|
||||
//! Instead, we work around this with things like the [`Accumulate`] trait.
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::binary::length_take;
|
||||
use crate::combinator::trace;
|
||||
use crate::stream::Accumulate;
|
||||
use crate::stream::Partial;
|
||||
use crate::stream::Stateful;
|
||||
+38
@@ -0,0 +1,38 @@
|
||||
//! # Chapter 0: Introduction
|
||||
//!
|
||||
//! This tutorial assumes that you are:
|
||||
//! - Already familiar with Rust
|
||||
//! - Using `winnow` for the first time
|
||||
//!
|
||||
//! The focus will be on parsing in-memory strings (`&str`). Once done, you might want to check the
|
||||
//! [Special Topics][_topic] for more specialized topics or examples.
|
||||
//!
|
||||
//! ## About
|
||||
//!
|
||||
//! `winnow` is a parser-combinator library. In other words, it gives you tools to define:
|
||||
//! - "parsers", or functions that take an input and give back an output
|
||||
//! - "combinators", or functions that take parsers and _combine_ them together!
|
||||
//!
|
||||
//! While "combinator" might be an unfamiliar word, you are likely using them in your rust code
|
||||
//! today, like with the [`Iterator`] trait:
|
||||
//! ```rust
|
||||
//! let data = vec![1, 2, 3, 4, 5];
|
||||
//! let even_count = data.iter()
|
||||
//! .copied() // combinator
|
||||
//! .filter(|d| d % 2 == 0) // combinator
|
||||
//! .count(); // combinator
|
||||
//! ```
|
||||
//!
|
||||
//! Parser combinators are great because:
|
||||
//!
|
||||
//! - Individual parser functions are small, focused on one thing, ignoring the rest
|
||||
//! - You can write tests focused on individual parsers (unit tests and property-based tests)
|
||||
//! in addition to testing the top-level parser as a whole.
|
||||
//! - Top-level parsing code looks close to the grammar you would have written
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::_topic;
|
||||
use std::iter::Iterator;
|
||||
|
||||
pub use super::chapter_1 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+88
@@ -0,0 +1,88 @@
|
||||
//! # Chapter 1: The Winnow Way
|
||||
//!
|
||||
//! First of all, we need to understand the way that winnow thinks about parsing.
|
||||
//! As discussed in the introduction, winnow lets us compose more complex parsers from more simple
|
||||
//! ones (using "combinators").
|
||||
//!
|
||||
//! Let's discuss what a "parser" actually does. A parser takes an input and advances it until it returns
|
||||
//! a result, where:
|
||||
//! - `Ok` indicates the parser successfully found what it was looking for; or
|
||||
//! - `Err` indicates the parser could not find what it was looking for.
|
||||
//!
|
||||
//! Parsers do more than just return a binary "success"/"failure" code.
|
||||
//! - On success, the parser will return the processed data. The input will be advanced to the end of
|
||||
//! what was processed, pointing to what will be parsed next.
|
||||
//! - If the parser failed, then there are multiple errors that could be returned.
|
||||
//! We'll explore this further in [`chapter_7`].
|
||||
//!
|
||||
//! ```text
|
||||
//! ┌─► Ok(what matched the parser)
|
||||
//! ┌────────┐ │
|
||||
//! my input───►│a parser├──►either──┤
|
||||
//! └────────┘ └─► Err(...)
|
||||
//! ```
|
||||
//!
|
||||
//!
|
||||
//! To represent this model of the world, winnow uses the [`Result<O>`] type.
|
||||
//! The `Ok` variant has `output: O`;
|
||||
//! whereas the `Err` variant stores an error.
|
||||
//!
|
||||
//! You can import that from:
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::Result;
|
||||
//! ```
|
||||
//!
|
||||
//! To combine parsers, we need a common way to refer to them which is where the [`Parser<I, O, E>`]
|
||||
//! trait comes in with [`Parser::parse_next`] being the primary way to drive
|
||||
//! parsing forward.
|
||||
//! In [`chapter_6`], we'll cover how to integrate these into your application, particularly with
|
||||
//! [`Parser::parse`].
|
||||
//!
|
||||
//! You'll note that `I` and `O` are parameterized -- while most of the examples in this book
|
||||
//! will be with `&str` (i.e. parsing a string); [they do not have to be strings][stream]; nor do they
|
||||
//! have to be the same type (consider the simple example where `I = &str`, and `O = u64` -- this
|
||||
//! parses a string into an unsigned integer.)
|
||||
//!
|
||||
//! # Let's write our first parser!
|
||||
//!
|
||||
//! The simplest parser we can write is one which successfully does nothing.
|
||||
//!
|
||||
//! To make it easier to implement a [`Parser`], the trait is implemented for
|
||||
//! functions of the form `Fn(&mut I) -> Result<O>`.
|
||||
//!
|
||||
//! This parser function should take in a `&str`:
|
||||
//!
|
||||
//! - Since it is supposed to succeed, we know it will return the `Ok` variant.
|
||||
//! - Since it does nothing to our input, the input will be left where it started.
|
||||
//! - Since it doesn't parse anything, it also should just return an empty string.
|
||||
//!
|
||||
//! ```rust
|
||||
//! use winnow::Result;
|
||||
//! use winnow::Parser;
|
||||
//!
|
||||
//! pub fn do_nothing_parser<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! Ok("")
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let output = do_nothing_parser.parse_next(&mut input).unwrap();
|
||||
//! // Same as:
|
||||
//! // let output = do_nothing_parser(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, "0x1a2b Hello");
|
||||
//! assert_eq!(output, "");
|
||||
//! }
|
||||
//! ```
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use super::chapter_6;
|
||||
use super::chapter_7;
|
||||
use crate::Parser;
|
||||
use crate::_topic::stream;
|
||||
|
||||
pub use super::chapter_0 as previous;
|
||||
pub use super::chapter_2 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+301
@@ -0,0 +1,301 @@
|
||||
//! # Chapter 2: Tokens and Tags
|
||||
//!
|
||||
//! The simplest *useful* parser you can write is one which matches tokens.
|
||||
//! In our case, tokens are `char`.
|
||||
//!
|
||||
//! ## Tokens
|
||||
//!
|
||||
//! [`Stream`] provides some core operations to help with parsing. For example, to process a
|
||||
//! single token, you can do:
|
||||
//! ```rust
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::stream::Stream;
|
||||
//! use winnow::error::ParserError;
|
||||
//!
|
||||
//! fn parse_prefix(input: &mut &str) -> Result<char> {
|
||||
//! let c = input.next_token().ok_or_else(|| {
|
||||
//! ParserError::from_input(input)
|
||||
//! })?;
|
||||
//! if c != '0' {
|
||||
//! return Err(ParserError::from_input(input));
|
||||
//! }
|
||||
//! Ok(c)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, "x1a2b Hello");
|
||||
//! assert_eq!(output, '0');
|
||||
//!
|
||||
//! assert!(parse_prefix.parse_next(&mut "d").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! This extraction of a token is encapsulated in the [`any`] parser:
|
||||
//! ```rust
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::error::ParserError;
|
||||
//! use winnow::Parser;
|
||||
//! use winnow::token::any;
|
||||
//!
|
||||
//! fn parse_prefix(input: &mut &str) -> Result<char> {
|
||||
//! let c = any
|
||||
//! .parse_next(input)?;
|
||||
//! if c != '0' {
|
||||
//! return Err(ParserError::from_input(input));
|
||||
//! }
|
||||
//! Ok(c)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, "x1a2b Hello");
|
||||
//! # assert_eq!(output, '0');
|
||||
//! #
|
||||
//! # assert!(parse_prefix.parse_next(&mut "d").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! Using the higher level [`any`] parser opens `parse_prefix` to the helpers on the [`Parser`] trait,
|
||||
//! like [`Parser::verify`] which fails a parse if a condition isn't met, like our check above:
|
||||
//! ```rust
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::Parser;
|
||||
//! use winnow::token::any;
|
||||
//!
|
||||
//! fn parse_prefix(input: &mut &str) -> Result<char> {
|
||||
//! let c = any
|
||||
//! .verify(|c| *c == '0')
|
||||
//! .parse_next(input)?;
|
||||
//! Ok(c)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, "x1a2b Hello");
|
||||
//! # assert_eq!(output, '0');
|
||||
//! #
|
||||
//! # assert!(parse_prefix.parse_next(&mut "d").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! Matching a single token literal is common enough that [`Parser`] is implemented for
|
||||
//! the `char` type, encapsulating both [`any`] and [`Parser::verify`]:
|
||||
//! ```rust
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::Parser;
|
||||
//!
|
||||
//! fn parse_prefix(input: &mut &str) -> Result<char> {
|
||||
//! let c = '0'.parse_next(input)?;
|
||||
//! Ok(c)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, "x1a2b Hello");
|
||||
//! # assert_eq!(output, '0');
|
||||
//! #
|
||||
//! # assert!(parse_prefix.parse_next(&mut "d").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! ## Tags
|
||||
//!
|
||||
//! [`Stream`] also supports processing slices of tokens:
|
||||
//! ```rust
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::stream::Stream;
|
||||
//! use winnow::error::ParserError;
|
||||
//!
|
||||
//! fn parse_prefix<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! let expected = "0x";
|
||||
//! if input.len() < expected.len() {
|
||||
//! return Err(ParserError::from_input(input));
|
||||
//! }
|
||||
//! let actual = input.next_slice(expected.len());
|
||||
//! if actual != expected {
|
||||
//! return Err(ParserError::from_input(input));
|
||||
//! }
|
||||
//! Ok(actual)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! assert_eq!(input, "1a2b Hello");
|
||||
//! assert_eq!(output, "0x");
|
||||
//!
|
||||
//! assert!(parse_prefix.parse_next(&mut "0o123").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Matching the input position against a string literal is encapsulated in the [`literal`] parser:
|
||||
//! ```rust
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::Parser;
|
||||
//! use winnow::token::literal;
|
||||
//!
|
||||
//! fn parse_prefix<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! let expected = "0x";
|
||||
//! let actual = literal(expected).parse_next(input)?;
|
||||
//! Ok(actual)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! # assert_eq!(input, "1a2b Hello");
|
||||
//! # assert_eq!(output, "0x");
|
||||
//! #
|
||||
//! # assert!(parse_prefix.parse_next(&mut "0o123").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! Like for a single token, matching a string literal is common enough that [`Parser`] is implemented for the `&str` type:
|
||||
//! ```rust
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::Parser;
|
||||
//!
|
||||
//! fn parse_prefix<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! let actual = "0x".parse_next(input)?;
|
||||
//! Ok(actual)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let output = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! # assert_eq!(input, "1a2b Hello");
|
||||
//! # assert_eq!(output, "0x");
|
||||
//! #
|
||||
//! # assert!(parse_prefix.parse_next(&mut "0o123").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! See [`token`] for additional individual and token-slice parsers.
|
||||
//!
|
||||
//! ## Character Classes
|
||||
//!
|
||||
//! Selecting a single `char` or a [`literal`] is fairly limited. Sometimes, you will want to select one of several
|
||||
//! `chars` of a specific class, like digits. For this, we use the [`one_of`] parser:
|
||||
//!
|
||||
//! ```rust
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::token::one_of;
|
||||
//!
|
||||
//! fn parse_digits(input: &mut &str) -> Result<char> {
|
||||
//! one_of(('0'..='9', 'a'..='f', 'A'..='F')).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "1a2b Hello";
|
||||
//!
|
||||
//! let output = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! assert_eq!(input, "a2b Hello");
|
||||
//! assert_eq!(output, '1');
|
||||
//!
|
||||
//! assert!(parse_digits.parse_next(&mut "Z").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! > **Aside:** [`one_of`] might look straightforward, a function returning a value that implements `Parser`.
|
||||
//! > Let's look at it more closely as its used above (resolving all generic parameters):
|
||||
//! > ```rust
|
||||
//! > # use winnow::prelude::*;
|
||||
//! > # use winnow::error::ContextError;
|
||||
//! > pub fn one_of<'i>(
|
||||
//! > list: &'static [char]
|
||||
//! > ) -> impl Parser<&'i str, char, ContextError> {
|
||||
//! > // ...
|
||||
//! > # winnow::token::one_of(list)
|
||||
//! > }
|
||||
//! > ```
|
||||
//! > If you have not programmed in a language where functions are values, the type signature of the
|
||||
//! > [`one_of`] function might be a surprise.
|
||||
//! > The function [`one_of`] *returns a function*. The function it returns is a
|
||||
//! > [`Parser`], taking a `&str` and returning an [`Result`]. This is a common pattern in winnow for
|
||||
//! > configurable or stateful parsers.
|
||||
//!
|
||||
//! Some of character classes are common enough that a named parser is provided, like with:
|
||||
//! - [`line_ending`][crate::ascii::line_ending]: Recognizes an end of line (both `\n` and `\r\n`)
|
||||
//! - [`newline`][crate::ascii::newline]: Matches a newline character `\n`
|
||||
//! - [`tab`][crate::ascii::tab]: Matches a tab character `\t`
|
||||
//!
|
||||
//! You can then capture sequences of these characters with parsers like [`take_while`].
|
||||
//! ```rust
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::token::take_while;
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! take_while(1.., ('0'..='9', 'a'..='f', 'A'..='F')).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "1a2b Hello";
|
||||
//!
|
||||
//! let output = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(output, "1a2b");
|
||||
//!
|
||||
//! assert!(parse_digits.parse_next(&mut "Z").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! We could simplify this further by using one of the built-in character classes, [`hex_digit1`]:
|
||||
//! ```rust
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::Result;
|
||||
//! use winnow::ascii::hex_digit1;
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! hex_digit1.parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "1a2b Hello";
|
||||
//!
|
||||
//! let output = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(output, "1a2b");
|
||||
//!
|
||||
//! assert!(parse_digits.parse_next(&mut "Z").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! See [`ascii`] for more text-based parsers.
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::ascii;
|
||||
use crate::ascii::hex_digit1;
|
||||
use crate::stream::ContainsToken;
|
||||
use crate::stream::Stream;
|
||||
use crate::token;
|
||||
use crate::token::any;
|
||||
use crate::token::literal;
|
||||
use crate::token::one_of;
|
||||
use crate::token::take_while;
|
||||
use crate::Parser;
|
||||
use crate::Result;
|
||||
use std::ops::RangeInclusive;
|
||||
|
||||
pub use super::chapter_1 as previous;
|
||||
pub use super::chapter_3 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+394
@@ -0,0 +1,394 @@
|
||||
//! # Chapter 3: Sequencing and Alternatives
|
||||
//!
|
||||
//! In the last chapter, we saw how to create parsers using prebuilt parsers.
|
||||
//!
|
||||
//! In this chapter, we explore two other widely used features:
|
||||
//! sequencing and alternatives.
|
||||
//!
|
||||
//! ## Sequencing
|
||||
//!
|
||||
//! Now that we can create more interesting parsers, we can sequence them together, like:
|
||||
//!
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! #
|
||||
//! fn parse_prefix<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! "0x".parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! take_while(1.., (
|
||||
//! ('0'..='9'),
|
||||
//! ('A'..='F'),
|
||||
//! ('a'..='f'),
|
||||
//! )).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let prefix = parse_prefix.parse_next(&mut input).unwrap();
|
||||
//! let digits = parse_digits.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(prefix, "0x");
|
||||
//! assert_eq!(digits, "1a2b");
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! To sequence these together, you can just put them in a tuple:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! #
|
||||
//! # fn parse_prefix<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # "0x".parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! //...
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let (prefix, digits) = (
|
||||
//! parse_prefix,
|
||||
//! parse_digits
|
||||
//! ).parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(prefix, "0x");
|
||||
//! assert_eq!(digits, "1a2b");
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Frequently, you won't care about the literal and you can instead use one of the provided combinators,
|
||||
//! like [`preceded`]:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! use winnow::combinator::preceded;
|
||||
//!
|
||||
//! # fn parse_prefix<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # "0x".parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! //...
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let digits = preceded(
|
||||
//! parse_prefix,
|
||||
//! parse_digits
|
||||
//! ).parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(digits, "1a2b");
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! See [`combinator`] for more sequencing parsers.
|
||||
//!
|
||||
//! ## Alternatives
|
||||
//!
|
||||
//! Sometimes, we might want to choose between two parsers; and we're happy with
|
||||
//! either being used.
|
||||
//!
|
||||
//! To retry a parse result, we can save off a [`Stream::checkpoint`] and later restore the parser
|
||||
//! back to that position with [`Stream::reset`]:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! use winnow::stream::Stream;
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! let start = input.checkpoint();
|
||||
//! if let Ok(output) = ("0b", parse_bin_digits).parse_next(input) {
|
||||
//! return Ok(output);
|
||||
//! }
|
||||
//!
|
||||
//! input.reset(&start);
|
||||
//! if let Ok(output) = ("0o", parse_oct_digits).parse_next(input) {
|
||||
//! return Ok(output);
|
||||
//! }
|
||||
//!
|
||||
//! input.reset(&start);
|
||||
//! if let Ok(output) = ("0d", parse_dec_digits).parse_next(input) {
|
||||
//! return Ok(output);
|
||||
//! }
|
||||
//!
|
||||
//! input.reset(&start);
|
||||
//! ("0x", parse_hex_digits).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let (prefix, digits) = parse_digits.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(prefix, "0x");
|
||||
//! assert_eq!(digits, "1a2b");
|
||||
//!
|
||||
//! assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! <div class="warning">
|
||||
//!
|
||||
//! **Warning:** the above example is for illustrative purposes and relying on `Result::Ok` or
|
||||
//! `Result::Err` can lead to incorrect behavior. This will be clarified in later when covering
|
||||
//! [error handling][`chapter_7`#error-cuts]
|
||||
//!
|
||||
//! </div>
|
||||
//!
|
||||
//! [`opt`] is a parser that encapsulates this pattern of "retry on failure":
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! use winnow::combinator::opt;
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! if let Some(output) = opt(("0b", parse_bin_digits)).parse_next(input)? {
|
||||
//! Ok(output)
|
||||
//! } else if let Some(output) = opt(("0o", parse_oct_digits)).parse_next(input)? {
|
||||
//! Ok(output)
|
||||
//! } else if let Some(output) = opt(("0d", parse_dec_digits)).parse_next(input)? {
|
||||
//! Ok(output)
|
||||
//! } else {
|
||||
//! ("0x", parse_hex_digits).parse_next(input)
|
||||
//! }
|
||||
//! }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let (prefix, digits) = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, " Hello");
|
||||
//! # assert_eq!(prefix, "0x");
|
||||
//! # assert_eq!(digits, "1a2b");
|
||||
//! #
|
||||
//! # assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! [`alt`] encapsulates this if/else-if ladder pattern, with the last case being the "else":
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! use winnow::combinator::alt;
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! alt((
|
||||
//! ("0b", parse_bin_digits),
|
||||
//! ("0o", parse_oct_digits),
|
||||
//! ("0d", parse_dec_digits),
|
||||
//! ("0x", parse_hex_digits),
|
||||
//! )).parse_next(input)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let (prefix, digits) = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, " Hello");
|
||||
//! # assert_eq!(prefix, "0x");
|
||||
//! # assert_eq!(digits, "1a2b");
|
||||
//! #
|
||||
//! # assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! <div class="warning">
|
||||
//!
|
||||
//! **Note:** [`empty`] and [`fail`] are parsers that might be useful in the "else" case.
|
||||
//!
|
||||
//! </div>
|
||||
//!
|
||||
//! Sometimes a giant if/else-if ladder can be slow and you'd rather have a `match` statement for
|
||||
//! branches of your parser that have unique prefixes. In this case, you can use the
|
||||
//! [`dispatch`] macro:
|
||||
//!
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! use winnow::combinator::dispatch;
|
||||
//! use winnow::token::take;
|
||||
//! use winnow::combinator::fail;
|
||||
//!
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! dispatch!(take(2usize);
|
||||
//! "0b" => parse_bin_digits,
|
||||
//! "0o" => parse_oct_digits,
|
||||
//! "0d" => parse_dec_digits,
|
||||
//! "0x" => parse_hex_digits,
|
||||
//! _ => fail,
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let digits = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, " Hello");
|
||||
//! # assert_eq!(digits, "1a2b");
|
||||
//! #
|
||||
//! # assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! <div class="warning">
|
||||
//!
|
||||
//! **Note:** [`peek`] may be useful when [`dispatch`]ing from hints from each case's parser.
|
||||
//!
|
||||
//! </div>
|
||||
//!
|
||||
//! See [`combinator`] for more alternative parsers.
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use super::chapter_7;
|
||||
use crate::combinator;
|
||||
use crate::combinator::alt;
|
||||
use crate::combinator::dispatch;
|
||||
use crate::combinator::empty;
|
||||
use crate::combinator::fail;
|
||||
use crate::combinator::opt;
|
||||
use crate::combinator::peek;
|
||||
use crate::combinator::preceded;
|
||||
use crate::stream::Stream;
|
||||
|
||||
pub use super::chapter_2 as previous;
|
||||
pub use super::chapter_4 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+109
@@ -0,0 +1,109 @@
|
||||
//! # Chapter 4: Parsers With Custom Return Types
|
||||
//!
|
||||
//! So far, we have seen mostly functions that take an `&str`, and return a
|
||||
//! [`Result<&str>`]. Splitting strings into smaller strings and characters is certainly
|
||||
//! useful, but it's not the only thing winnow is capable of!
|
||||
//!
|
||||
//! A useful operation when parsing is to convert between types; for example
|
||||
//! parsing from `&str` to another primitive, like [`usize`].
|
||||
//!
|
||||
//! All we need to do for our parser to return a different type is to change
|
||||
//! the type parameter of [`Result`] to the desired return type.
|
||||
//! For example, to return a `usize`, return a `Result<usize>`.
|
||||
//!
|
||||
//! One winnow-native way of doing a type conversion is to use the
|
||||
//! [`Parser::parse_to`] combinator
|
||||
//! to convert from a successful parse to a particular type using [`FromStr`].
|
||||
//!
|
||||
//! The following code converts from a string containing a number to `usize`:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::ascii::digit1;
|
||||
//! #
|
||||
//! fn parse_digits(input: &mut &str) -> Result<usize> {
|
||||
//! digit1
|
||||
//! .parse_to()
|
||||
//! .parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "1024 Hello";
|
||||
//!
|
||||
//! let output = parse_digits.parse_next(&mut input).unwrap();
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(output, 1024);
|
||||
//!
|
||||
//! assert!(parse_digits(&mut "Z").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! `Parser::parse_to` is just a convenient form of [`Parser::try_map`] which we can use to handle
|
||||
//! all radices of numbers:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! use winnow::combinator::dispatch;
|
||||
//! use winnow::token::take;
|
||||
//! use winnow::combinator::fail;
|
||||
//!
|
||||
//! fn parse_digits(input: &mut &str) -> Result<usize> {
|
||||
//! dispatch!(take(2usize);
|
||||
//! "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
|
||||
//! "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
|
||||
//! "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
|
||||
//! "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
|
||||
//! _ => fail,
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b Hello";
|
||||
//!
|
||||
//! let digits = parse_digits.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(digits, 0x1a2b);
|
||||
//!
|
||||
//! assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! See also [`Parser`] for more output-modifying parsers.
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use crate::Parser;
|
||||
use crate::Result;
|
||||
use std::str::FromStr;
|
||||
|
||||
pub use super::chapter_3 as previous;
|
||||
pub use super::chapter_5 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+300
@@ -0,0 +1,300 @@
|
||||
//! # Chapter 5: Repetition
|
||||
//!
|
||||
//! In [`chapter_3`], we covered how to sequence different parsers into a tuple but sometimes you need to run a
|
||||
//! single parser multiple times, collecting the result into a container, like `Vec`.
|
||||
//!
|
||||
//! Let's collect the result of `parse_digits`:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::dispatch;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! use winnow::combinator::opt;
|
||||
//! use winnow::combinator::repeat;
|
||||
//! use winnow::combinator::terminated;
|
||||
//!
|
||||
//! fn parse_list(input: &mut &str) -> Result<Vec<usize>> {
|
||||
//! let mut list = Vec::new();
|
||||
//! while let Some(output) = opt(terminated(parse_digits, opt(','))).parse_next(input)? {
|
||||
//! list.push(output);
|
||||
//! }
|
||||
//! Ok(list)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//! # fn parse_digits(input: &mut &str) -> Result<usize> {
|
||||
//! # dispatch!(take(2usize);
|
||||
//! # "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
|
||||
//! # "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
|
||||
//! # "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
|
||||
//! # "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
|
||||
//! # _ => fail,
|
||||
//! # ).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b,0x3c4d,0x5e6f Hello";
|
||||
//!
|
||||
//! let digits = parse_list.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(digits, vec![0x1a2b, 0x3c4d, 0x5e6f]);
|
||||
//!
|
||||
//! assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! We can implement this declaratively with [`repeat`]:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::dispatch;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! use winnow::combinator::opt;
|
||||
//! use winnow::combinator::repeat;
|
||||
//! use winnow::combinator::terminated;
|
||||
//!
|
||||
//! fn parse_list(input: &mut &str) -> Result<Vec<usize>> {
|
||||
//! repeat(0..,
|
||||
//! terminated(parse_digits, opt(','))
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn parse_digits(input: &mut &str) -> Result<usize> {
|
||||
//! # dispatch!(take(2usize);
|
||||
//! # "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
|
||||
//! # "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
|
||||
//! # "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
|
||||
//! # "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
|
||||
//! # _ => fail,
|
||||
//! # ).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b,0x3c4d,0x5e6f Hello";
|
||||
//! #
|
||||
//! # let digits = parse_list.parse_next(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, " Hello");
|
||||
//! # assert_eq!(digits, vec![0x1a2b, 0x3c4d, 0x5e6f]);
|
||||
//! #
|
||||
//! # assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! You'll notice that the above allows trailing `,`. However, if that's not desired, it
|
||||
//! can easily be fixed by using [`separated`] instead of [`repeat`]:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::dispatch;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! use winnow::combinator::separated;
|
||||
//!
|
||||
//! fn parse_list(input: &mut &str) -> Result<Vec<usize>> {
|
||||
//! separated(0.., parse_digits, ",").parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//! # fn parse_digits(input: &mut &str) -> Result<usize> {
|
||||
//! # dispatch!(take(2usize);
|
||||
//! # "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
|
||||
//! # "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
|
||||
//! # "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
|
||||
//! # "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
|
||||
//! # _ => fail,
|
||||
//! # ).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b,0x3c4d,0x5e6f Hello";
|
||||
//!
|
||||
//! let digits = parse_list.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(digits, vec![0x1a2b, 0x3c4d, 0x5e6f]);
|
||||
//!
|
||||
//! assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! If you look closely at [`separated`] and [`repeat`], they aren't limited to collecting
|
||||
//! the result into a `Vec`, but rather anything that implements the [`Accumulate`] trait.
|
||||
//! [`Accumulate`] is for instance also implemented for [`HashSet`], [`String`] and `()`.
|
||||
//!
|
||||
//! This lets us build more complex parsers than we did in
|
||||
//! [`chapter_2`] by accumulating the results into a `()` and [`take`][Parser::take]-ing
|
||||
//! the consumed input.
|
||||
//!
|
||||
//! `take` works by
|
||||
//! 1. Creating a [`checkpoint`][Stream::checkpoint]
|
||||
//! 2. Running the inner parser, in our case the `parse_list` parser, which will advance the input
|
||||
//! 3. Returning the slice from the first checkpoint to the current position.
|
||||
//!
|
||||
//! Since the result of `parse_list` gets thrown away, we
|
||||
//! accumulates into a `()` to not waste work creating an unused `Vec`.
|
||||
//!
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::dispatch;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::combinator::separated;
|
||||
//! #
|
||||
//! fn take_list<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! parse_list.take().parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! fn parse_list(input: &mut &str) -> Result<()> {
|
||||
//! separated(0.., parse_digits, ",").parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! # fn parse_digits(input: &mut &str) -> Result<usize> {
|
||||
//! # dispatch!(take(2usize);
|
||||
//! # "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
|
||||
//! # "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
|
||||
//! # "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
|
||||
//! # "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
|
||||
//! # _ => fail,
|
||||
//! # ).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let mut input = "0x1a2b,0x3c4d,0x5e6f Hello";
|
||||
//!
|
||||
//! let digits = take_list.parse_next(&mut input).unwrap();
|
||||
//!
|
||||
//! assert_eq!(input, " Hello");
|
||||
//! assert_eq!(digits, "0x1a2b,0x3c4d,0x5e6f");
|
||||
//!
|
||||
//! assert!(parse_digits(&mut "ghiWorld").is_err());
|
||||
//! }
|
||||
//! ```
|
||||
//! See [`combinator`] for more repetition parsers.
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use super::chapter_2;
|
||||
use super::chapter_3;
|
||||
use crate::combinator;
|
||||
use crate::combinator::repeat;
|
||||
use crate::combinator::separated;
|
||||
use crate::stream::Accumulate;
|
||||
use crate::stream::Stream;
|
||||
use crate::Parser;
|
||||
use std::collections::HashSet;
|
||||
|
||||
pub use super::chapter_4 as previous;
|
||||
pub use super::chapter_6 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+120
@@ -0,0 +1,120 @@
|
||||
//! # Chapter 6: Integrating the Parser
|
||||
//!
|
||||
//! So far, we've highlighted how to incrementally parse, but how do we bring this all together
|
||||
//! into our application?
|
||||
//!
|
||||
//! Parsers we've been working with look like:
|
||||
//! ```rust
|
||||
//! # use winnow::error::ContextError;
|
||||
//! # use winnow::Parser;
|
||||
//! use winnow::Result;
|
||||
//!
|
||||
//! pub fn parser<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! // ...
|
||||
//! # Ok("")
|
||||
//! }
|
||||
//! ```
|
||||
//! 1. We have to decide what to do about the "remainder" of the `input`.
|
||||
//! 2. The [`Result`] may not be compatible with the rest of the Rust ecosystem.
|
||||
//! Normally, Rust applications want errors that are `std::error::Error + Send + Sync + 'static`
|
||||
//! meaning:
|
||||
//! - They implement the [`std::error::Error`] trait
|
||||
//! - They can be sent across threads
|
||||
//! - They are safe to be referenced across threads
|
||||
//! - They do not borrow
|
||||
//!
|
||||
//! winnow provides [`Parser::parse`] to help with this:
|
||||
//! - Ensures we hit [`eof`]
|
||||
//! - Wraps the error in [`ParseError`]
|
||||
//! - For simple cases, [`ParseError`] provides a [`std::fmt::Display`] impl to render the error.
|
||||
//! - For more involved cases, [`ParseError`] provides the original [`input`][ParseError::input] and the
|
||||
//! [`offset`][ParseError::offset] of where it failed so you can capture this information in
|
||||
//! your error, [rendering it as you wish][chapter_7#error-adaptation-and-rendering].
|
||||
//! - Converts from [`ModalResult`] to [`Result`] (if used, more on this in [`chapter_7`])
|
||||
//!
|
||||
//! However, [`ParseError`] will still need some level of adaptation to integrate with your
|
||||
//! application's error type (like with `?`).
|
||||
//!
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::dispatch;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! use winnow::Parser;
|
||||
//!
|
||||
//! #[derive(Debug, PartialEq, Eq)]
|
||||
//! pub struct Hex(usize);
|
||||
//!
|
||||
//! impl std::str::FromStr for Hex {
|
||||
//! type Err = anyhow::Error;
|
||||
//!
|
||||
//! fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! parse_digits
|
||||
//! .map(Hex)
|
||||
//! .parse(input)
|
||||
//! .map_err(|e| anyhow::format_err!("{e}"))
|
||||
//! }
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> Result<usize> {
|
||||
//! # dispatch!(take(2usize);
|
||||
//! # "0b" => parse_bin_digits.try_map(|s| usize::from_str_radix(s, 2)),
|
||||
//! # "0o" => parse_oct_digits.try_map(|s| usize::from_str_radix(s, 8)),
|
||||
//! # "0d" => parse_dec_digits.try_map(|s| usize::from_str_radix(s, 10)),
|
||||
//! # "0x" => parse_hex_digits.try_map(|s| usize::from_str_radix(s, 16)),
|
||||
//! # _ => fail,
|
||||
//! # ).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//!
|
||||
//! fn main() {
|
||||
//! let input = "0x1a2b";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap(), Hex(0x1a2b));
|
||||
//!
|
||||
//! let input = "0x1a2b Hello";
|
||||
//! assert!(input.parse::<Hex>().is_err());
|
||||
//! let input = "ghiHello";
|
||||
//! assert!(input.parse::<Hex>().is_err());
|
||||
//! }
|
||||
//! ```
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use super::chapter_1;
|
||||
use super::chapter_7;
|
||||
use crate::combinator::eof;
|
||||
use crate::error::ErrMode;
|
||||
use crate::error::ParseError;
|
||||
use crate::ModalResult;
|
||||
use crate::Parser;
|
||||
|
||||
pub use super::chapter_5 as previous;
|
||||
pub use super::chapter_7 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+736
@@ -0,0 +1,736 @@
|
||||
//! # Chapter 7: Error Reporting
|
||||
//!
|
||||
//! ## Context
|
||||
//!
|
||||
//! With [`Parser::parse`] we get errors that point to the failure but don't explain the reason for
|
||||
//! the failure:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! #
|
||||
//! # #[derive(Debug, PartialEq, Eq)]
|
||||
//! # pub struct Hex(usize);
|
||||
//! #
|
||||
//! # impl std::str::FromStr for Hex {
|
||||
//! # type Err = String;
|
||||
//! #
|
||||
//! # fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! # .parse(input)
|
||||
//! # .map_err(|e| e.to_string())
|
||||
//! # }
|
||||
//! # }
|
||||
//! #
|
||||
//! // ...
|
||||
//!
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! # alt((
|
||||
//! # ("0b", parse_bin_digits),
|
||||
//! # ("0o", parse_oct_digits),
|
||||
//! # ("0d", parse_dec_digits),
|
||||
//! # ("0x", parse_hex_digits),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "0xZZ";
|
||||
//! let error = "\
|
||||
//! 0xZZ
|
||||
//! ^
|
||||
//! ";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Back in [`chapter_1`], we glossed over the `Err` variant of [`Result`]. `Result<O>` is
|
||||
//! actually short for `Result<O, E=ContextError>` where [`ContextError`] is a relatively cheap
|
||||
//! way of building up reasonable errors for humans.
|
||||
//!
|
||||
//! You can use [`Parser::context`] to annotate the error with custom types
|
||||
//! while unwinding to further clarify the error:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! use winnow::error::StrContext;
|
||||
//! use winnow::error::StrContextValue;
|
||||
//!
|
||||
//! #
|
||||
//! # #[derive(Debug, PartialEq, Eq)]
|
||||
//! # pub struct Hex(usize);
|
||||
//! #
|
||||
//! # impl std::str::FromStr for Hex {
|
||||
//! # type Err = String;
|
||||
//! #
|
||||
//! # fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! # .parse(input)
|
||||
//! # .map_err(|e| e.to_string())
|
||||
//! # }
|
||||
//! # }
|
||||
//! #
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! alt((
|
||||
//! ("0b", parse_bin_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("binary"))),
|
||||
//! ("0o", parse_oct_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("octal"))),
|
||||
//! ("0d", parse_dec_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("decimal"))),
|
||||
//! ("0x", parse_hex_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("hexadecimal"))),
|
||||
//! )).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//!
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "0xZZ";
|
||||
//! let error = "\
|
||||
//! 0xZZ
|
||||
//! ^
|
||||
//! invalid digit
|
||||
//! expected hexadecimal";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! If you remember back to [`chapter_3`], [`alt`] will only report the last error.
|
||||
//! So if the parsers fail for any reason, like a bad radix, it will be reported as an invalid
|
||||
//! hexadecimal value:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::error::StrContext;
|
||||
//! # use winnow::error::StrContextValue;
|
||||
//! #
|
||||
//! #
|
||||
//! # #[derive(Debug, PartialEq, Eq)]
|
||||
//! # pub struct Hex(usize);
|
||||
//! #
|
||||
//! # impl std::str::FromStr for Hex {
|
||||
//! # type Err = String;
|
||||
//! #
|
||||
//! # fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! # .parse(input)
|
||||
//! # .map_err(|e| e.to_string())
|
||||
//! # }
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! # alt((
|
||||
//! # ("0b", parse_bin_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("binary"))),
|
||||
//! # ("0o", parse_oct_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("octal"))),
|
||||
//! # ("0d", parse_dec_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("decimal"))),
|
||||
//! # ("0x", parse_hex_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("hexadecimal"))),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "100";
|
||||
//! let error = "\
|
||||
//! 100
|
||||
//! ^
|
||||
//! invalid digit
|
||||
//! expected hexadecimal";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//! We can improve this with [`fail`]:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! use winnow::error::StrContext;
|
||||
//! use winnow::error::StrContextValue;
|
||||
//!
|
||||
//! #
|
||||
//! # #[derive(Debug, PartialEq, Eq)]
|
||||
//! # pub struct Hex(usize);
|
||||
//! #
|
||||
//! # impl std::str::FromStr for Hex {
|
||||
//! # type Err = String;
|
||||
//! #
|
||||
//! # fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! # .parse(input)
|
||||
//! # .map_err(|e| e.to_string())
|
||||
//! # }
|
||||
//! # }
|
||||
//! #
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! alt((
|
||||
//! ("0b", parse_bin_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("binary"))),
|
||||
//! ("0o", parse_oct_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("octal"))),
|
||||
//! ("0d", parse_dec_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("decimal"))),
|
||||
//! ("0x", parse_hex_digits)
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("hexadecimal"))),
|
||||
//! fail
|
||||
//! .context(StrContext::Label("radix prefix"))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0b")))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0o")))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0d")))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0x"))),
|
||||
//! )).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//!
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "100";
|
||||
//! let error = "\
|
||||
//! 100
|
||||
//! ^
|
||||
//! invalid radix prefix
|
||||
//! expected `0b`, `0o`, `0d`, `0x`";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! ## Error Cuts
|
||||
//!
|
||||
//! We still have the issue that we are falling-through when the radix is valid but the digits
|
||||
//! don't match it:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::Result;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::error::StrContext;
|
||||
//! # use winnow::error::StrContextValue;
|
||||
//! #
|
||||
//! #
|
||||
//! # #[derive(Debug, PartialEq, Eq)]
|
||||
//! # pub struct Hex(usize);
|
||||
//! #
|
||||
//! # impl std::str::FromStr for Hex {
|
||||
//! # type Err = String;
|
||||
//! #
|
||||
//! # fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! # .parse(input)
|
||||
//! # .map_err(|e| e.to_string())
|
||||
//! # }
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> Result<(&'s str, &'s str)> {
|
||||
//! # alt((
|
||||
//! # ("0b", parse_bin_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("binary"))),
|
||||
//! # ("0o", parse_oct_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("octal"))),
|
||||
//! # ("0d", parse_dec_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("decimal"))),
|
||||
//! # ("0x", parse_hex_digits)
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("hexadecimal"))),
|
||||
//! # fail
|
||||
//! # .context(StrContext::Label("radix prefix"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0b")))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0o")))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0d")))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0x"))),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> Result<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "0b5";
|
||||
//! let error = "\
|
||||
//! 0b5
|
||||
//! ^
|
||||
//! invalid radix prefix
|
||||
//! expected `0b`, `0o`, `0d`, `0x`";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! Winnow provides an error wrapper, [`ErrMode<ContextError>`], so different failure modes can affect parsing.
|
||||
//! [`ErrMode`] is an enum with [`Backtrack`] and [`Cut`] variants (ignore [`Incomplete`] as its only
|
||||
//! relevant for [streaming][_topic::stream]). By default, errors are [`Backtrack`], meaning that
|
||||
//! other parsing branches will be attempted on failure, like the next case of an [`alt`]. [`Cut`]
|
||||
//! shortcircuits all other branches, immediately reporting the error.
|
||||
//!
|
||||
//! To make [`ErrMode`] more convenient, Winnow provides [`ModalResult`]:
|
||||
//! ```rust
|
||||
//! # use winnow::error::ContextError;
|
||||
//! # use winnow::error::ErrMode;
|
||||
//! pub type ModalResult<O, E = ContextError> = Result<O, ErrMode<E>>;
|
||||
//! ```
|
||||
//!
|
||||
//! So we can get the correct `context` by changing to [`ModalResult`] and adding [`cut_err`]:
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::error::StrContext;
|
||||
//! # use winnow::error::StrContextValue;
|
||||
//! use winnow::combinator::cut_err;
|
||||
//!
|
||||
//! #
|
||||
//! # #[derive(Debug, PartialEq, Eq)]
|
||||
//! # pub struct Hex(usize);
|
||||
//! #
|
||||
//! # impl std::str::FromStr for Hex {
|
||||
//! # type Err = String;
|
||||
//! #
|
||||
//! # fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! # .parse(input)
|
||||
//! # .map_err(|e| e.to_string())
|
||||
//! # }
|
||||
//! # }
|
||||
//! #
|
||||
//! fn parse_digits<'s>(input: &mut &'s str) -> ModalResult<(&'s str, &'s str)> {
|
||||
//! alt((
|
||||
//! ("0b", cut_err(parse_bin_digits))
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("binary"))),
|
||||
//! ("0o", cut_err(parse_oct_digits))
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("octal"))),
|
||||
//! ("0d", cut_err(parse_dec_digits))
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("decimal"))),
|
||||
//! ("0x", cut_err(parse_hex_digits))
|
||||
//! .context(StrContext::Label("digit"))
|
||||
//! .context(StrContext::Expected(StrContextValue::Description("hexadecimal"))),
|
||||
//! fail
|
||||
//! .context(StrContext::Label("radix prefix"))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0b")))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0o")))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0d")))
|
||||
//! .context(StrContext::Expected(StrContextValue::StringLiteral("0x"))),
|
||||
//! )).parse_next(input)
|
||||
//! }
|
||||
//!
|
||||
//! // ...
|
||||
//!
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "0b5";
|
||||
//! let error = "\
|
||||
//! 0b5
|
||||
//! ^
|
||||
//! invalid digit
|
||||
//! expected binary";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! ## Error Adaptation and Rendering
|
||||
//!
|
||||
//! While Winnow can provide basic rendering of errors, your application can have various demands
|
||||
//! beyond the basics provided like
|
||||
//! - Correctly reporting columns with unicode
|
||||
//! - Conforming to a specific layout
|
||||
//!
|
||||
//! For example, to get rustc-like errors with [`annotate-snippets`](https://crates.io/crates/annotate-snippets):
|
||||
//! ```rust
|
||||
//! # use winnow::prelude::*;
|
||||
//! # use winnow::token::take_while;
|
||||
//! # use winnow::combinator::alt;
|
||||
//! # use winnow::token::take;
|
||||
//! # use winnow::combinator::fail;
|
||||
//! # use winnow::Parser;
|
||||
//! # use winnow::error::ParseError;
|
||||
//! # use winnow::error::ContextError;
|
||||
//! # use winnow::error::StrContext;
|
||||
//! # use winnow::error::StrContextValue;
|
||||
//! # use winnow::combinator::cut_err;
|
||||
//! #
|
||||
//! #
|
||||
//! #[derive(Debug, PartialEq, Eq)]
|
||||
//! pub struct Hex(usize);
|
||||
//!
|
||||
//! impl std::str::FromStr for Hex {
|
||||
//! type Err = HexError;
|
||||
//!
|
||||
//! fn from_str(input: &str) -> Result<Self, Self::Err> {
|
||||
//! // ...
|
||||
//! # parse_digits
|
||||
//! # .try_map(|(t, v)| match t {
|
||||
//! # "0b" => usize::from_str_radix(v, 2),
|
||||
//! # "0o" => usize::from_str_radix(v, 8),
|
||||
//! # "0d" => usize::from_str_radix(v, 10),
|
||||
//! # "0x" => usize::from_str_radix(v, 16),
|
||||
//! # _ => unreachable!("`parse_digits` doesn't return `{t}`"),
|
||||
//! # })
|
||||
//! # .map(Hex)
|
||||
//! .parse(input)
|
||||
//! .map_err(|e| HexError::from_parse(e))
|
||||
//! }
|
||||
//! }
|
||||
//!
|
||||
//! #[derive(Debug)]
|
||||
//! pub struct HexError {
|
||||
//! message: String,
|
||||
//! // Byte spans are tracked, rather than line and column.
|
||||
//! // This makes it easier to operate on programmatically
|
||||
//! // and doesn't limit us to one definition for column count
|
||||
//! // which can depend on the output medium and application.
|
||||
//! span: std::ops::Range<usize>,
|
||||
//! input: String,
|
||||
//! }
|
||||
//!
|
||||
//! impl HexError {
|
||||
//! // Avoiding `From` so `winnow` types don't become part of our public API
|
||||
//! fn from_parse(error: ParseError<&str, ContextError>) -> Self {
|
||||
//! // The default renderer for `ContextError` is still used but that can be
|
||||
//! // customized as well to better fit your needs.
|
||||
//! let message = error.inner().to_string();
|
||||
//! let input = (*error.input()).to_owned();
|
||||
//! // Assume the error span is only for the first `char`.
|
||||
//! let span = error.char_span();
|
||||
//! Self {
|
||||
//! message,
|
||||
//! span,
|
||||
//! input,
|
||||
//! }
|
||||
//! }
|
||||
//! }
|
||||
//!
|
||||
//! impl std::fmt::Display for HexError {
|
||||
//! fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
||||
//! let message = annotate_snippets::Level::Error.title(&self.message)
|
||||
//! .snippet(annotate_snippets::Snippet::source(&self.input)
|
||||
//! .fold(true)
|
||||
//! .annotation(annotate_snippets::Level::Error.span(self.span.clone()))
|
||||
//! );
|
||||
//! let renderer = annotate_snippets::Renderer::plain();
|
||||
//! let rendered = renderer.render(message);
|
||||
//! rendered.fmt(f)
|
||||
//! }
|
||||
//! }
|
||||
//!
|
||||
//! impl std::error::Error for HexError {}
|
||||
//!
|
||||
//! # fn parse_digits<'s>(input: &mut &'s str) -> ModalResult<(&'s str, &'s str)> {
|
||||
//! # alt((
|
||||
//! # ("0b", cut_err(parse_bin_digits))
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("binary"))),
|
||||
//! # ("0o", cut_err(parse_oct_digits))
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("octal"))),
|
||||
//! # ("0d", cut_err(parse_dec_digits))
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("decimal"))),
|
||||
//! # ("0x", cut_err(parse_hex_digits))
|
||||
//! # .context(StrContext::Label("digit"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::Description("hexadecimal"))),
|
||||
//! # fail
|
||||
//! # .context(StrContext::Label("radix prefix"))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0b")))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0o")))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0d")))
|
||||
//! # .context(StrContext::Expected(StrContextValue::StringLiteral("0x"))),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_bin_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='1'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_oct_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='7'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_dec_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! #
|
||||
//! # fn parse_hex_digits<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! # take_while(1.., (
|
||||
//! # ('0'..='9'),
|
||||
//! # ('A'..='F'),
|
||||
//! # ('a'..='f'),
|
||||
//! # )).parse_next(input)
|
||||
//! # }
|
||||
//! fn main() {
|
||||
//! let input = "0b5";
|
||||
//! let error = "\
|
||||
//! error: invalid digit
|
||||
//! expected binary
|
||||
//! |
|
||||
//! 1 | 0b5
|
||||
//! | ^
|
||||
//! |";
|
||||
//! assert_eq!(input.parse::<Hex>().unwrap_err().to_string(), error);
|
||||
//! }
|
||||
//! ```
|
||||
//!
|
||||
//! To add spans to your parsed data for inclusion in semantic errors, see [`Parser::with_span`].
|
||||
//!
|
||||
//! For richer syntactic errors with spans,
|
||||
//! consider separating lexing and parsing and annotating your tokens with [`Parser::with_span`].
|
||||
|
||||
#![allow(unused_imports)]
|
||||
use super::chapter_1;
|
||||
use super::chapter_3;
|
||||
use crate::combinator::alt;
|
||||
use crate::combinator::cut_err;
|
||||
use crate::combinator::fail;
|
||||
use crate::error::ContextError;
|
||||
use crate::error::ErrMode;
|
||||
use crate::error::ErrMode::*;
|
||||
use crate::ModalResult;
|
||||
use crate::Parser;
|
||||
use crate::Result;
|
||||
use crate::_topic;
|
||||
|
||||
pub use super::chapter_6 as previous;
|
||||
pub use super::chapter_8 as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+37
@@ -0,0 +1,37 @@
|
||||
//! # Chapter 8: Debugging
|
||||
//!
|
||||
//! When things inevitably go wrong, you can introspect the parsing state by running your test case
|
||||
//! with `--features winnow/debug`.
|
||||
//!
|
||||
//! For example, the trace output of an [escaped string parser][crate::_topic::language#escaped-strings]:
|
||||
//! 
|
||||
//!
|
||||
//! You can extend your own parsers to show up by wrapping their body with
|
||||
//! [`trace`][crate::combinator::trace]. Going back to [`do_nothing_parser`][super::chapter_1].
|
||||
//! ```rust
|
||||
//! # use winnow::ModalResult;
|
||||
//! # use winnow::Parser;
|
||||
//! use winnow::combinator::trace;
|
||||
//!
|
||||
//! pub fn do_nothing_parser<'s>(input: &mut &'s str) -> ModalResult<&'s str> {
|
||||
//! trace(
|
||||
//! "do_nothing_parser",
|
||||
//! |i: &mut _| Ok("")
|
||||
//! ).parse_next(input)
|
||||
//! }
|
||||
//! #
|
||||
//! # fn main() {
|
||||
//! # let mut input = "0x1a2b Hello";
|
||||
//! #
|
||||
//! # let output = do_nothing_parser.parse_next(&mut input).unwrap();
|
||||
//! # // Same as:
|
||||
//! # // let output = do_nothing_parser(&mut input).unwrap();
|
||||
//! #
|
||||
//! # assert_eq!(input, "0x1a2b Hello");
|
||||
//! # assert_eq!(output, "");
|
||||
//! # }
|
||||
//! ```
|
||||
|
||||
pub use super::chapter_7 as previous;
|
||||
pub use crate::_topic as next;
|
||||
pub use crate::_tutorial as table_of_contents;
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
//! # Tutorial
|
||||
//!
|
||||
//! Table of Contents
|
||||
#![allow(clippy::std_instead_of_core)]
|
||||
|
||||
pub mod chapter_0;
|
||||
pub mod chapter_1;
|
||||
pub mod chapter_2;
|
||||
pub mod chapter_3;
|
||||
pub mod chapter_4;
|
||||
pub mod chapter_5;
|
||||
pub mod chapter_6;
|
||||
pub mod chapter_7;
|
||||
pub mod chapter_8;
|
||||
+1864
File diff suppressed because it is too large.
Load diff
+4658
File diff suppressed because it is too large.
Load diff
+398
@@ -0,0 +1,398 @@
|
||||
//! Bit level parsers
|
||||
//!
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
use crate::combinator::trace;
|
||||
use crate::error::{ErrorConvert, Needed, ParserError};
|
||||
use crate::stream::{Stream, StreamIsPartial, ToUsize};
|
||||
use crate::{Parser, Result};
|
||||
use core::ops::{AddAssign, Div, Shl, Shr};
|
||||
|
||||
/// Number of bits in a byte
|
||||
const BYTE: usize = u8::BITS as usize;
|
||||
|
||||
/// Converts a byte-level input to a bit-level input
|
||||
///
|
||||
/// See [`bytes`] to convert it back.
|
||||
///
|
||||
/// # Example
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::Bytes;
|
||||
/// # use winnow::binary::bits::{bits, take};
|
||||
/// # use winnow::error::ContextError;
|
||||
/// # use winnow::error::ErrMode;
|
||||
/// type Stream<'i> = &'i Bytes;
|
||||
///
|
||||
/// fn stream(b: &[u8]) -> Stream<'_> {
|
||||
/// Bytes::new(b)
|
||||
/// }
|
||||
///
|
||||
/// fn parse(input: &mut Stream<'_>) -> ModalResult<(u8, u8)> {
|
||||
/// bits::<_, _, ErrMode<ContextError>, _, _>((take(4usize), take(8usize))).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// let input = stream(&[0x12, 0x34, 0xff, 0xff]);
|
||||
///
|
||||
/// let output = parse.parse_peek(input).expect("We take 1.5 bytes and the input is longer than 2 bytes");
|
||||
///
|
||||
/// // The first byte is consumed, the second byte is partially consumed and dropped.
|
||||
/// let remaining = output.0;
|
||||
/// assert_eq!(remaining, stream(&[0xff, 0xff]));
|
||||
///
|
||||
/// let parsed = output.1;
|
||||
/// assert_eq!(parsed.0, 0x01);
|
||||
/// assert_eq!(parsed.1, 0x23);
|
||||
/// ```
|
||||
pub fn bits<Input, Output, BitError, ByteError, ParseNext>(
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Output, ByteError>
|
||||
where
|
||||
BitError: ParserError<(Input, usize)> + ErrorConvert<ByteError>,
|
||||
ByteError: ParserError<Input>,
|
||||
(Input, usize): Stream,
|
||||
Input: Stream + Clone,
|
||||
ParseNext: Parser<(Input, usize), Output, BitError>,
|
||||
{
|
||||
trace("bits", move |input: &mut Input| {
|
||||
let mut bit_input = (input.clone(), 0);
|
||||
match parser.parse_next(&mut bit_input) {
|
||||
Ok(result) => {
|
||||
let (mut rest, offset) = bit_input;
|
||||
// If the next byte has been partially read, it will be sliced away as well.
|
||||
// The parser functions might already slice away all fully read bytes.
|
||||
// That's why `offset / BYTE` isn't necessarily needed at all times.
|
||||
let remaining_bytes_index = offset / BYTE + if offset % BYTE == 0 { 0 } else { 1 };
|
||||
let _ = rest.next_slice(remaining_bytes_index);
|
||||
*input = rest;
|
||||
Ok(result)
|
||||
}
|
||||
Err(e) => match e.needed() {
|
||||
Some(n) => Err(ParserError::incomplete(
|
||||
input,
|
||||
n.map(|u| u.get() / BYTE + 1),
|
||||
)),
|
||||
None => Err(ErrorConvert::convert(e)),
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Convert a [`bits`] stream back into a byte stream
|
||||
///
|
||||
/// <div class="warning">
|
||||
///
|
||||
/// **Warning:** A partial byte remaining in the input will be ignored and the given parser will
|
||||
/// start parsing at the next full byte.
|
||||
///
|
||||
/// </div>
|
||||
///
|
||||
/// # Examples
|
||||
///
|
||||
/// ```
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::Bytes;
|
||||
/// # use winnow::token::rest;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// # use winnow::error::ErrMode;
|
||||
/// use winnow::binary::bits::{bits, bytes, take};
|
||||
///
|
||||
/// type Stream<'i> = &'i Bytes;
|
||||
///
|
||||
/// fn stream(b: &[u8]) -> Stream<'_> {
|
||||
/// Bytes::new(b)
|
||||
/// }
|
||||
///
|
||||
/// fn parse<'i>(input: &mut Stream<'i>) -> ModalResult<(u8, u8, &'i [u8])> {
|
||||
/// bits::<_, _, ErrMode<ContextError>, _, _>((
|
||||
/// take(4usize),
|
||||
/// take(8usize),
|
||||
/// bytes::<_, _, ErrMode<ContextError>, _, _>(rest)
|
||||
/// )).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// let input = stream(&[0x12, 0x34, 0xff, 0xff]);
|
||||
///
|
||||
/// assert_eq!(parse.parse_peek(input), Ok(( stream(&[]), (0x01, 0x23, &[0xff, 0xff][..]) )));
|
||||
/// ```
|
||||
pub fn bytes<Input, Output, ByteError, BitError, ParseNext>(
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<(Input, usize), Output, BitError>
|
||||
where
|
||||
ByteError: ParserError<Input> + ErrorConvert<BitError>,
|
||||
BitError: ParserError<(Input, usize)>,
|
||||
Input: Stream<Token = u8> + Clone,
|
||||
ParseNext: Parser<Input, Output, ByteError>,
|
||||
{
|
||||
trace("bytes", move |bit_input: &mut (Input, usize)| {
|
||||
let (mut input, offset) = bit_input.clone();
|
||||
let _ = if offset % BYTE != 0 {
|
||||
input.next_slice(1 + offset / BYTE)
|
||||
} else {
|
||||
input.next_slice(offset / BYTE)
|
||||
};
|
||||
match parser.parse_next(&mut input) {
|
||||
Ok(res) => {
|
||||
*bit_input = (input, 0);
|
||||
Ok(res)
|
||||
}
|
||||
Err(e) => match e.needed() {
|
||||
Some(Needed::Unknown) => Err(ParserError::incomplete(bit_input, Needed::Unknown)),
|
||||
Some(Needed::Size(sz)) => Err(match sz.get().checked_mul(BYTE) {
|
||||
Some(v) => ParserError::incomplete(bit_input, Needed::new(v)),
|
||||
None => ParserError::assert(
|
||||
bit_input,
|
||||
"overflow in turning needed bytes into needed bits",
|
||||
),
|
||||
}),
|
||||
None => Err(ErrorConvert::convert(e)),
|
||||
},
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Parse taking `count` bits
|
||||
///
|
||||
/// # Effective Signature
|
||||
///
|
||||
/// Assuming you are parsing a `(&[u8], usize)` bit [Stream]:
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// pub fn take<'i>(count: usize) -> impl Parser<(&'i [u8], usize), u8, ContextError>
|
||||
/// # {
|
||||
/// # winnow::binary::bits::take(count)
|
||||
/// # }
|
||||
/// ```
|
||||
///
|
||||
/// # Example
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::Bytes;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// use winnow::binary::bits::take;
|
||||
///
|
||||
/// type Stream<'i> = &'i Bytes;
|
||||
///
|
||||
/// fn stream(b: &[u8]) -> Stream<'_> {
|
||||
/// Bytes::new(b)
|
||||
/// }
|
||||
///
|
||||
/// // Consumes 0 bits, returns 0
|
||||
/// assert_eq!(take::<_, usize, _, ContextError>(0usize).parse_peek((stream(&[0b00010010]), 0)), Ok(((stream(&[0b00010010]), 0), 0)));
|
||||
///
|
||||
/// // Consumes 4 bits, returns their values and increase offset to 4
|
||||
/// assert_eq!(take::<_, usize, _, ContextError>(4usize).parse_peek((stream(&[0b00010010]), 0)), Ok(((stream(&[0b00010010]), 4), 0b00000001)));
|
||||
///
|
||||
/// // Consumes 4 bits, offset is 4, returns their values and increase offset to 0 of next byte
|
||||
/// assert_eq!(take::<_, usize, _, ContextError>(4usize).parse_peek((stream(&[0b00010010]), 4)), Ok(((stream(&[]), 0), 0b00000010)));
|
||||
///
|
||||
/// // Tries to consume 12 bits but only 8 are available
|
||||
/// assert!(take::<_, usize, _, ContextError>(12usize).parse_peek((stream(&[0b00010010]), 0)).is_err());
|
||||
/// ```
|
||||
#[inline(always)]
|
||||
pub fn take<Input, Output, Count, Error>(count: Count) -> impl Parser<(Input, usize), Output, Error>
|
||||
where
|
||||
Input: Stream<Token = u8> + StreamIsPartial + Clone,
|
||||
Output: From<u8> + AddAssign + Shl<usize, Output = Output> + Shr<usize, Output = Output>,
|
||||
Count: ToUsize,
|
||||
Error: ParserError<(Input, usize)>,
|
||||
{
|
||||
let count = count.to_usize();
|
||||
trace("take", move |input: &mut (Input, usize)| {
|
||||
if <Input as StreamIsPartial>::is_partial_supported() {
|
||||
take_::<_, _, _, true>(input, count)
|
||||
} else {
|
||||
take_::<_, _, _, false>(input, count)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
fn take_<I, O, E: ParserError<(I, usize)>, const PARTIAL: bool>(
|
||||
bit_input: &mut (I, usize),
|
||||
count: usize,
|
||||
) -> Result<O, E>
|
||||
where
|
||||
I: StreamIsPartial,
|
||||
I: Stream<Token = u8> + Clone,
|
||||
O: From<u8> + AddAssign + Shl<usize, Output = O> + Shr<usize, Output = O>,
|
||||
{
|
||||
if count == 0 {
|
||||
Ok(0u8.into())
|
||||
} else {
|
||||
let (mut input, bit_offset) = bit_input.clone();
|
||||
if input.eof_offset() * BYTE < count + bit_offset {
|
||||
if PARTIAL && input.is_partial() {
|
||||
Err(ParserError::incomplete(bit_input, Needed::new(count)))
|
||||
} else {
|
||||
Err(ParserError::from_input(&(input, bit_offset)))
|
||||
}
|
||||
} else {
|
||||
let cnt = (count + bit_offset).div(BYTE);
|
||||
let mut acc: O = 0_u8.into();
|
||||
let mut offset: usize = bit_offset;
|
||||
let mut remaining: usize = count;
|
||||
let mut end_offset: usize = 0;
|
||||
|
||||
for (_, byte) in input.iter_offsets().take(cnt + 1) {
|
||||
if remaining == 0 {
|
||||
break;
|
||||
}
|
||||
let val: O = if offset == 0 {
|
||||
byte.into()
|
||||
} else {
|
||||
(byte << offset >> offset).into()
|
||||
};
|
||||
|
||||
if remaining < BYTE - offset {
|
||||
acc += val >> (BYTE - offset - remaining);
|
||||
end_offset = remaining + offset;
|
||||
break;
|
||||
} else {
|
||||
acc += val << (remaining - (BYTE - offset));
|
||||
remaining -= BYTE - offset;
|
||||
offset = 0;
|
||||
}
|
||||
}
|
||||
let _ = input.next_slice(cnt);
|
||||
*bit_input = (input, end_offset);
|
||||
Ok(acc)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Parse taking `count` bits and comparing them to `pattern`
|
||||
///
|
||||
/// # Effective Signature
|
||||
///
|
||||
/// Assuming you are parsing a `(&[u8], usize)` bit [Stream]:
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// pub fn pattern<'i>(pattern: u8, count: usize) -> impl Parser<(&'i [u8], usize), u8, ContextError>
|
||||
/// # {
|
||||
/// # winnow::binary::bits::pattern(pattern, count)
|
||||
/// # }
|
||||
/// ```
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::Bytes;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// use winnow::binary::bits::pattern;
|
||||
///
|
||||
/// type Stream<'i> = &'i Bytes;
|
||||
///
|
||||
/// fn stream(b: &[u8]) -> Stream<'_> {
|
||||
/// Bytes::new(b)
|
||||
/// }
|
||||
///
|
||||
/// /// Compare the lowest `count` bits of `input` against the lowest `count` bits of `pattern`.
|
||||
/// /// Return Ok and the matching section of `input` if there's a match.
|
||||
/// /// Return Err if there's no match.
|
||||
/// fn parser(bits: u8, count: u8, input: &mut (Stream<'_>, usize)) -> ModalResult<u8> {
|
||||
/// pattern(bits, count).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// // The lowest 4 bits of 0b00001111 match the lowest 4 bits of 0b11111111.
|
||||
/// assert_eq!(
|
||||
/// pattern::<_, usize, _, ContextError>(0b0000_1111, 4usize).parse_peek((stream(&[0b1111_1111]), 0)),
|
||||
/// Ok(((stream(&[0b1111_1111]), 4), 0b0000_1111))
|
||||
/// );
|
||||
///
|
||||
/// // The lowest bit of 0b00001111 matches the lowest bit of 0b11111111 (both are 1).
|
||||
/// assert_eq!(
|
||||
/// pattern::<_, usize, _, ContextError>(0b00000001, 1usize).parse_peek((stream(&[0b11111111]), 0)),
|
||||
/// Ok(((stream(&[0b11111111]), 1), 0b00000001))
|
||||
/// );
|
||||
///
|
||||
/// // The lowest 2 bits of 0b11111111 and 0b00000001 are different.
|
||||
/// assert!(pattern::<_, usize, _, ContextError>(0b000000_01, 2usize).parse_peek((stream(&[0b111111_11]), 0)).is_err());
|
||||
///
|
||||
/// // The lowest 8 bits of 0b11111111 and 0b11111110 are different.
|
||||
/// assert!(pattern::<_, usize, _, ContextError>(0b11111110, 8usize).parse_peek((stream(&[0b11111111]), 0)).is_err());
|
||||
/// ```
|
||||
#[inline(always)]
|
||||
#[doc(alias = "literal")]
|
||||
#[doc(alias = "just")]
|
||||
#[doc(alias = "tag")]
|
||||
pub fn pattern<Input, Output, Count, Error: ParserError<(Input, usize)>>(
|
||||
pattern: Output,
|
||||
count: Count,
|
||||
) -> impl Parser<(Input, usize), Output, Error>
|
||||
where
|
||||
Input: Stream<Token = u8> + StreamIsPartial + Clone,
|
||||
Count: ToUsize,
|
||||
Output: From<u8>
|
||||
+ AddAssign
|
||||
+ Shl<usize, Output = Output>
|
||||
+ Shr<usize, Output = Output>
|
||||
+ PartialEq,
|
||||
{
|
||||
let count = count.to_usize();
|
||||
trace("pattern", move |input: &mut (Input, usize)| {
|
||||
let start = input.checkpoint();
|
||||
|
||||
take(count).parse_next(input).and_then(|o| {
|
||||
if pattern == o {
|
||||
Ok(o)
|
||||
} else {
|
||||
input.reset(&start);
|
||||
Err(ParserError::from_input(input))
|
||||
}
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
/// Parses one specific bit as a bool.
|
||||
///
|
||||
/// # Effective Signature
|
||||
///
|
||||
/// Assuming you are parsing a `(&[u8], usize)` bit [Stream]:
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// pub fn bool(input: &mut (&[u8], usize)) -> ModalResult<bool>
|
||||
/// # {
|
||||
/// # winnow::binary::bits::bool.parse_next(input)
|
||||
/// # }
|
||||
/// ```
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::Bytes;
|
||||
/// # use winnow::error::InputError;
|
||||
/// use winnow::binary::bits::bool;
|
||||
///
|
||||
/// type Stream<'i> = &'i Bytes;
|
||||
///
|
||||
/// fn stream(b: &[u8]) -> Stream<'_> {
|
||||
/// Bytes::new(b)
|
||||
/// }
|
||||
///
|
||||
/// fn parse(input: &mut (Stream<'_>, usize)) -> ModalResult<bool> {
|
||||
/// bool.parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parse.parse_peek((stream(&[0b10000000]), 0)), Ok(((stream(&[0b10000000]), 1), true)));
|
||||
/// assert_eq!(parse.parse_peek((stream(&[0b10000000]), 1)), Ok(((stream(&[0b10000000]), 2), false)));
|
||||
/// ```
|
||||
#[doc(alias = "any")]
|
||||
pub fn bool<Input, Error: ParserError<(Input, usize)>>(
|
||||
input: &mut (Input, usize),
|
||||
) -> Result<bool, Error>
|
||||
where
|
||||
Input: Stream<Token = u8> + StreamIsPartial + Clone,
|
||||
{
|
||||
trace("bool", |input: &mut (Input, usize)| {
|
||||
let bit: u32 = take(1usize).parse_next(input)?;
|
||||
Ok(bit != 0)
|
||||
})
|
||||
.parse_next(input)
|
||||
}
|
||||
+204
@@ -0,0 +1,204 @@
|
||||
use super::*;
|
||||
use crate::error::ErrMode;
|
||||
use crate::error::InputError;
|
||||
use crate::prelude::*;
|
||||
use crate::Partial;
|
||||
|
||||
#[test]
|
||||
/// Take the `bits` function and assert that remaining bytes are correctly returned, if the
|
||||
/// previous bytes are fully consumed
|
||||
fn test_complete_byte_consumption_bits() {
|
||||
let input = &[0x12, 0x34, 0x56, 0x78][..];
|
||||
|
||||
// Take 3 bit slices with sizes [4, 8, 4].
|
||||
#[allow(clippy::type_complexity)]
|
||||
let result: ModalResult<(&[u8], (u8, u8, u8)), InputError<_>> =
|
||||
bits::<_, _, ErrMode<InputError<(&[u8], usize)>>, _, _>((
|
||||
take(4usize),
|
||||
take(8usize),
|
||||
take(4usize),
|
||||
))
|
||||
.parse_peek(input);
|
||||
|
||||
let output = result.expect("We take 2 bytes and the input is longer than 2 bytes");
|
||||
|
||||
let remaining = output.0;
|
||||
assert_eq!(remaining, [0x56, 0x78]);
|
||||
|
||||
let parsed = output.1;
|
||||
assert_eq!(parsed.0, 0x01);
|
||||
assert_eq!(parsed.1, 0x23);
|
||||
assert_eq!(parsed.2, 0x04);
|
||||
}
|
||||
|
||||
#[test]
|
||||
/// Take the `bits` function and assert that remaining bytes are correctly returned, if the
|
||||
/// previous bytes are NOT fully consumed. Partially consumed bytes are supposed to be dropped.
|
||||
/// I.e. if we consume 1.5 bytes of 4 bytes, 2 bytes will be returned, bits 13-16 will be
|
||||
/// dropped.
|
||||
fn test_partial_byte_consumption_bits() {
|
||||
let input = &[0x12, 0x34, 0x56, 0x78][..];
|
||||
|
||||
// Take bit slices with sizes [4, 8].
|
||||
let result: ModalResult<(&[u8], (u8, u8)), InputError<_>> =
|
||||
bits::<_, _, ErrMode<InputError<(&[u8], usize)>>, _, _>((take(4usize), take(8usize)))
|
||||
.parse_peek(input);
|
||||
|
||||
let output = result.expect("We take 1.5 bytes and the input is longer than 2 bytes");
|
||||
|
||||
let remaining = output.0;
|
||||
assert_eq!(remaining, [0x56, 0x78]);
|
||||
|
||||
let parsed = output.1;
|
||||
assert_eq!(parsed.0, 0x01);
|
||||
assert_eq!(parsed.1, 0x23);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg(feature = "std")]
|
||||
/// Ensure that in Incomplete error is thrown, if too few bytes are passed for a given parser.
|
||||
fn test_incomplete_bits() {
|
||||
let input = Partial::new(&[0x12][..]);
|
||||
|
||||
// Take bit slices with sizes [4, 8].
|
||||
let result: ModalResult<(_, (u8, u8)), InputError<_>> =
|
||||
bits::<_, _, ErrMode<InputError<(_, usize)>>, _, _>((take(4usize), take(8usize)))
|
||||
.parse_peek(input);
|
||||
|
||||
assert!(result.is_err());
|
||||
let error = result.err().unwrap();
|
||||
assert_eq!("Parsing requires 2 more data", error.to_string());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_take_complete_0() {
|
||||
let input = &[0b00010010][..];
|
||||
let count = 0usize;
|
||||
assert_eq!(count, 0usize);
|
||||
let offset = 0usize;
|
||||
|
||||
let result: ModalResult<((&[u8], usize), usize), InputError<_>> =
|
||||
take(count).parse_peek((input, offset));
|
||||
|
||||
assert_eq!(result, Ok(((input, offset), 0)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_take_complete_eof() {
|
||||
let input = &[0b00010010][..];
|
||||
|
||||
let result: ModalResult<((&[u8], usize), usize), InputError<_>> =
|
||||
take(1usize).parse_peek((input, 8));
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
Err(crate::error::ErrMode::Backtrack(
|
||||
InputError::at((input, 8),)
|
||||
))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_take_complete_span_over_multiple_bytes() {
|
||||
let input = &[0b00010010, 0b00110100, 0b11111111, 0b11111111][..];
|
||||
|
||||
let result: ModalResult<((&[u8], usize), usize), InputError<_>> =
|
||||
take(24usize).parse_peek((input, 4));
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
Ok((([0b11111111].as_ref(), 4), 0b1000110100111111111111))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_take_partial_0() {
|
||||
let input = Partial::new(&[][..]);
|
||||
let count = 0usize;
|
||||
assert_eq!(count, 0usize);
|
||||
let offset = 0usize;
|
||||
|
||||
let result: ModalResult<((_, usize), usize), InputError<_>> =
|
||||
take(count).parse_peek((input, offset));
|
||||
|
||||
assert_eq!(result, Ok(((input, offset), 0)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pattern_partial_ok() {
|
||||
let input = Partial::new(&[0b00011111][..]);
|
||||
let offset = 0usize;
|
||||
let bits_to_take = 4usize;
|
||||
let value_to_pattern = 0b0001;
|
||||
|
||||
let result: ModalResult<((_, usize), usize), InputError<_>> =
|
||||
pattern(value_to_pattern, bits_to_take).parse_peek((input, offset));
|
||||
|
||||
assert_eq!(result, Ok(((input, bits_to_take), value_to_pattern)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pattern_partial_err() {
|
||||
let input = Partial::new(&[0b00011111][..]);
|
||||
let offset = 0usize;
|
||||
let bits_to_take = 4usize;
|
||||
let value_to_pattern = 0b1111;
|
||||
|
||||
let result: ModalResult<((_, usize), usize), InputError<_>> =
|
||||
pattern(value_to_pattern, bits_to_take).parse_peek((input, offset));
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
Err(crate::error::ErrMode::Backtrack(InputError::at((
|
||||
input, offset
|
||||
),)))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bool_0_complete() {
|
||||
let input = [0b10000000].as_ref();
|
||||
|
||||
let result: ModalResult<((&[u8], usize), bool), InputError<_>> = bool.parse_peek((input, 0));
|
||||
|
||||
assert_eq!(result, Ok(((input, 1), true)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bool_eof_complete() {
|
||||
let input = [0b10000000].as_ref();
|
||||
|
||||
let result: ModalResult<((&[u8], usize), bool), InputError<_>> = bool.parse_peek((input, 8));
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
Err(crate::error::ErrMode::Backtrack(
|
||||
InputError::at((input, 8),)
|
||||
))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bool_0_partial() {
|
||||
let input = Partial::new([0b10000000].as_ref());
|
||||
|
||||
#[allow(clippy::type_complexity)]
|
||||
let result: ModalResult<((Partial<&[u8]>, usize), bool), InputError<_>> =
|
||||
bool.parse_peek((input, 0));
|
||||
|
||||
assert_eq!(result, Ok(((input, 1), true)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_bool_eof_partial() {
|
||||
let input = Partial::new([0b10000000].as_ref());
|
||||
|
||||
#[allow(clippy::type_complexity)]
|
||||
let result: ModalResult<((Partial<&[u8]>, usize), bool), InputError<_>> =
|
||||
bool.parse_peek((input, 8));
|
||||
|
||||
assert_eq!(
|
||||
result,
|
||||
Err(crate::error::ErrMode::Incomplete(Needed::new(1)))
|
||||
);
|
||||
}
|
||||
+2533
File diff suppressed because it is too large.
Load diff
+3718
File diff suppressed because it is too large.
Load diff
+316
@@ -0,0 +1,316 @@
|
||||
use crate::combinator::trace;
|
||||
use crate::error::ParserError;
|
||||
use crate::stream::Stream;
|
||||
use crate::*;
|
||||
|
||||
#[doc(inline)]
|
||||
pub use crate::dispatch;
|
||||
|
||||
/// Helper trait for the [`alt()`] combinator.
|
||||
///
|
||||
/// This trait is implemented for tuples of up to 21 elements
|
||||
pub trait Alt<I, O, E> {
|
||||
/// Tests each parser in the tuple and returns the result of the first one that succeeds
|
||||
fn choice(&mut self, input: &mut I) -> Result<O, E>;
|
||||
}
|
||||
|
||||
/// Pick the first successful parser
|
||||
///
|
||||
/// To stop on an error, rather than trying further cases, see
|
||||
/// [`cut_err`][crate::combinator::cut_err] ([example][crate::_tutorial::chapter_7]).
|
||||
///
|
||||
/// For tight control over the error when no match is found, add a final case using [`fail`][crate::combinator::fail].
|
||||
/// Alternatively, with a [custom error type][crate::_topic::error], it is possible to track all
|
||||
/// errors or return the error of the parser that went the farthest in the input data.
|
||||
///
|
||||
/// When the alternative cases have unique prefixes, [`dispatch`] can offer better performance.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::ascii::{alpha1, digit1};
|
||||
/// use winnow::combinator::alt;
|
||||
/// # fn main() {
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// alt((alpha1, digit1)).parse_next(input)
|
||||
/// };
|
||||
///
|
||||
/// // the first parser, alpha1, takes the input
|
||||
/// assert_eq!(parser.parse_peek("abc"), Ok(("", "abc")));
|
||||
///
|
||||
/// // the first parser returns an error, so alt tries the second one
|
||||
/// assert_eq!(parser.parse_peek("123456"), Ok(("", "123456")));
|
||||
///
|
||||
/// // both parsers failed, and with the default error type, alt will return the last error
|
||||
/// assert!(parser.parse_peek(" ").is_err());
|
||||
/// # }
|
||||
/// ```
|
||||
#[doc(alias = "choice")]
|
||||
#[inline(always)]
|
||||
pub fn alt<Input: Stream, Output, Error, Alternatives>(
|
||||
mut alternatives: Alternatives,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Alternatives: Alt<Input, Output, Error>,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
trace("alt", move |i: &mut Input| alternatives.choice(i))
|
||||
}
|
||||
|
||||
/// Helper trait for the [`permutation()`] combinator.
|
||||
///
|
||||
/// This trait is implemented for tuples of up to 21 elements
|
||||
pub trait Permutation<I, O, E> {
|
||||
/// Tries to apply all parsers in the tuple in various orders until all of them succeed
|
||||
fn permutation(&mut self, input: &mut I) -> Result<O, E>;
|
||||
}
|
||||
|
||||
/// Deprecated, replaced with [`unordered_seq!`][crate::combinator::unordered_seq].
|
||||
#[deprecated(since = "0.7.14", note = "replaced with `unordered_seq!`")]
|
||||
#[inline(always)]
|
||||
pub fn permutation<I: Stream, O, E: ParserError<I>, List: Permutation<I, O, E>>(
|
||||
mut l: List,
|
||||
) -> impl Parser<I, O, E> {
|
||||
trace("permutation", move |i: &mut I| l.permutation(i))
|
||||
}
|
||||
|
||||
impl<const N: usize, I: Stream, O, E: ParserError<I>, P: Parser<I, O, E>> Alt<I, O, E> for [P; N] {
|
||||
fn choice(&mut self, input: &mut I) -> Result<O, E> {
|
||||
let mut error: Option<E> = None;
|
||||
|
||||
let start = input.checkpoint();
|
||||
for branch in self {
|
||||
input.reset(&start);
|
||||
match branch.parse_next(input) {
|
||||
Err(e) if e.is_backtrack() => {
|
||||
error = match error {
|
||||
Some(error) => Some(error.or(e)),
|
||||
None => Some(e),
|
||||
};
|
||||
}
|
||||
res => return res,
|
||||
}
|
||||
}
|
||||
|
||||
match error {
|
||||
Some(e) => Err(e.append(input, &start)),
|
||||
None => Err(ParserError::assert(
|
||||
input,
|
||||
"`alt` needs at least one parser",
|
||||
)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream, O, E: ParserError<I>, P: Parser<I, O, E>> Alt<I, O, E> for &mut [P] {
|
||||
fn choice(&mut self, input: &mut I) -> Result<O, E> {
|
||||
let mut error: Option<E> = None;
|
||||
|
||||
let start = input.checkpoint();
|
||||
for branch in self.iter_mut() {
|
||||
input.reset(&start);
|
||||
match branch.parse_next(input) {
|
||||
Err(e) if e.is_backtrack() => {
|
||||
error = match error {
|
||||
Some(error) => Some(error.or(e)),
|
||||
None => Some(e),
|
||||
};
|
||||
}
|
||||
res => return res,
|
||||
}
|
||||
}
|
||||
|
||||
match error {
|
||||
Some(e) => Err(e.append(input, &start)),
|
||||
None => Err(ParserError::assert(
|
||||
input,
|
||||
"`alt` needs at least one parser",
|
||||
)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! alt_trait(
|
||||
($first:ident $second:ident $($id: ident)+) => (
|
||||
alt_trait!(__impl $first $second; $($id)+);
|
||||
);
|
||||
(__impl $($current:ident)*; $head:ident $($id: ident)+) => (
|
||||
alt_trait_impl!($($current)*);
|
||||
|
||||
alt_trait!(__impl $($current)* $head; $($id)+);
|
||||
);
|
||||
(__impl $($current:ident)*; $head:ident) => (
|
||||
alt_trait_impl!($($current)*);
|
||||
alt_trait_impl!($($current)* $head);
|
||||
);
|
||||
);
|
||||
|
||||
macro_rules! alt_trait_impl(
|
||||
($($id:ident)+) => (
|
||||
impl<
|
||||
I: Stream, Output, Error: ParserError<I>,
|
||||
$($id: Parser<I, Output, Error>),+
|
||||
> Alt<I, Output, Error> for ( $($id),+ ) {
|
||||
|
||||
fn choice(&mut self, input: &mut I) -> Result<Output, Error> {
|
||||
let start = input.checkpoint();
|
||||
match self.0.parse_next(input) {
|
||||
Err(e) if e.is_backtrack() => alt_trait_inner!(1, self, input, start, e, $($id)+),
|
||||
res => res,
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
);
|
||||
|
||||
macro_rules! succ (
|
||||
(0, $submac:ident ! ($($rest:tt)*)) => ($submac!(1, $($rest)*));
|
||||
(1, $submac:ident ! ($($rest:tt)*)) => ($submac!(2, $($rest)*));
|
||||
(2, $submac:ident ! ($($rest:tt)*)) => ($submac!(3, $($rest)*));
|
||||
(3, $submac:ident ! ($($rest:tt)*)) => ($submac!(4, $($rest)*));
|
||||
(4, $submac:ident ! ($($rest:tt)*)) => ($submac!(5, $($rest)*));
|
||||
(5, $submac:ident ! ($($rest:tt)*)) => ($submac!(6, $($rest)*));
|
||||
(6, $submac:ident ! ($($rest:tt)*)) => ($submac!(7, $($rest)*));
|
||||
(7, $submac:ident ! ($($rest:tt)*)) => ($submac!(8, $($rest)*));
|
||||
(8, $submac:ident ! ($($rest:tt)*)) => ($submac!(9, $($rest)*));
|
||||
(9, $submac:ident ! ($($rest:tt)*)) => ($submac!(10, $($rest)*));
|
||||
(10, $submac:ident ! ($($rest:tt)*)) => ($submac!(11, $($rest)*));
|
||||
(11, $submac:ident ! ($($rest:tt)*)) => ($submac!(12, $($rest)*));
|
||||
(12, $submac:ident ! ($($rest:tt)*)) => ($submac!(13, $($rest)*));
|
||||
(13, $submac:ident ! ($($rest:tt)*)) => ($submac!(14, $($rest)*));
|
||||
(14, $submac:ident ! ($($rest:tt)*)) => ($submac!(15, $($rest)*));
|
||||
(15, $submac:ident ! ($($rest:tt)*)) => ($submac!(16, $($rest)*));
|
||||
(16, $submac:ident ! ($($rest:tt)*)) => ($submac!(17, $($rest)*));
|
||||
(17, $submac:ident ! ($($rest:tt)*)) => ($submac!(18, $($rest)*));
|
||||
(18, $submac:ident ! ($($rest:tt)*)) => ($submac!(19, $($rest)*));
|
||||
(19, $submac:ident ! ($($rest:tt)*)) => ($submac!(20, $($rest)*));
|
||||
(20, $submac:ident ! ($($rest:tt)*)) => ($submac!(21, $($rest)*));
|
||||
);
|
||||
|
||||
macro_rules! alt_trait_inner(
|
||||
($it:tt, $self:expr, $input:expr, $start:ident, $err:expr, $head:ident $($id:ident)+) => ({
|
||||
$input.reset(&$start);
|
||||
match $self.$it.parse_next($input) {
|
||||
Err(e) if e.is_backtrack() => {
|
||||
let err = $err.or(e);
|
||||
succ!($it, alt_trait_inner!($self, $input, $start, err, $($id)+))
|
||||
}
|
||||
res => res,
|
||||
}
|
||||
});
|
||||
($it:tt, $self:expr, $input:expr, $start:ident, $err:expr, $head:ident) => ({
|
||||
Err($err.append($input, &$start))
|
||||
});
|
||||
);
|
||||
|
||||
alt_trait!(Alt2 Alt3 Alt4 Alt5 Alt6 Alt7 Alt8 Alt9 Alt10 Alt11 Alt12 Alt13 Alt14 Alt15 Alt16 Alt17 Alt18 Alt19 Alt20 Alt21 Alt22);
|
||||
|
||||
// Manually implement Alt for (A,), the 1-tuple type
|
||||
impl<I: Stream, O, E: ParserError<I>, A: Parser<I, O, E>> Alt<I, O, E> for (A,) {
|
||||
fn choice(&mut self, input: &mut I) -> Result<O, E> {
|
||||
self.0.parse_next(input)
|
||||
}
|
||||
}
|
||||
|
||||
macro_rules! permutation_trait(
|
||||
(
|
||||
$name1:ident $ty1:ident $item1:ident
|
||||
$name2:ident $ty2:ident $item2:ident
|
||||
$($name3:ident $ty3:ident $item3:ident)*
|
||||
) => (
|
||||
permutation_trait!(__impl $name1 $ty1 $item1, $name2 $ty2 $item2; $($name3 $ty3 $item3)*);
|
||||
);
|
||||
(
|
||||
__impl $($name:ident $ty:ident $item:ident),+;
|
||||
$name1:ident $ty1:ident $item1:ident $($name2:ident $ty2:ident $item2:ident)*
|
||||
) => (
|
||||
permutation_trait_impl!($($name $ty $item),+);
|
||||
permutation_trait!(__impl $($name $ty $item),+ , $name1 $ty1 $item1; $($name2 $ty2 $item2)*);
|
||||
);
|
||||
(__impl $($name:ident $ty:ident $item:ident),+;) => (
|
||||
permutation_trait_impl!($($name $ty $item),+);
|
||||
);
|
||||
);
|
||||
|
||||
macro_rules! permutation_trait_impl(
|
||||
($($name:ident $ty:ident $item:ident),+) => (
|
||||
impl<
|
||||
I: Stream, $($ty),+ , Error: ParserError<I>,
|
||||
$($name: Parser<I, $ty, Error>),+
|
||||
> Permutation<I, ( $($ty),+ ), Error> for ( $($name),+ ) {
|
||||
|
||||
fn permutation(&mut self, input: &mut I) -> Result<( $($ty),+ ), Error> {
|
||||
let mut res = ($(Option::<$ty>::None),+);
|
||||
|
||||
loop {
|
||||
let mut err: Option<Error> = None;
|
||||
let start = input.checkpoint();
|
||||
permutation_trait_inner!(0, self, input, start, res, err, $($name)+);
|
||||
|
||||
// If we reach here, every iterator has either been applied before,
|
||||
// or errored on the remaining input
|
||||
if let Some(err) = err {
|
||||
// There are remaining parsers, and all errored on the remaining input
|
||||
input.reset(&start);
|
||||
return Err(err.append(input, &start));
|
||||
}
|
||||
|
||||
// All parsers were applied
|
||||
match res {
|
||||
($(Some($item)),+) => return Ok(($($item),+)),
|
||||
_ => unreachable!(),
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
);
|
||||
);
|
||||
|
||||
macro_rules! permutation_trait_inner(
|
||||
($it:tt, $self:expr, $input:ident, $start:ident, $res:expr, $err:expr, $head:ident $($id:ident)*) => (
|
||||
if $res.$it.is_none() {
|
||||
$input.reset(&$start);
|
||||
match $self.$it.parse_next($input) {
|
||||
Ok(o) => {
|
||||
$res.$it = Some(o);
|
||||
continue;
|
||||
}
|
||||
Err(e) if e.is_backtrack() => {
|
||||
$err = Some(match $err {
|
||||
Some(err) => err.or(e),
|
||||
None => e,
|
||||
});
|
||||
}
|
||||
Err(e) => return Err(e),
|
||||
};
|
||||
}
|
||||
succ!($it, permutation_trait_inner!($self, $input, $start, $res, $err, $($id)*));
|
||||
);
|
||||
($it:tt, $self:expr, $input:ident, $start:ident, $res:expr, $err:expr,) => ();
|
||||
);
|
||||
|
||||
permutation_trait!(
|
||||
P1 O1 o1
|
||||
P2 O2 o2
|
||||
P3 O3 o3
|
||||
P4 O4 o4
|
||||
P5 O5 o5
|
||||
P6 O6 o6
|
||||
P7 O7 o7
|
||||
P8 O8 o8
|
||||
P9 O9 o9
|
||||
P10 O10 o10
|
||||
P11 O11 o11
|
||||
P12 O12 o12
|
||||
P13 O13 o13
|
||||
P14 O14 o14
|
||||
P15 O15 o15
|
||||
P16 O16 o16
|
||||
P17 O17 o17
|
||||
P18 O18 o18
|
||||
P19 O19 o19
|
||||
P20 O20 o20
|
||||
P21 O21 o21
|
||||
);
|
||||
+506
@@ -0,0 +1,506 @@
|
||||
use crate::combinator::trace;
|
||||
use crate::error::{ModalError, ParserError};
|
||||
use crate::stream::Stream;
|
||||
use crate::*;
|
||||
|
||||
/// Apply a [`Parser`], producing `None` on [`ErrMode::Backtrack`][crate::error::ErrMode::Backtrack].
|
||||
///
|
||||
/// To chain an error up, see [`cut_err`].
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::opt;
|
||||
/// use winnow::ascii::alpha1;
|
||||
/// # fn main() {
|
||||
///
|
||||
/// fn parser<'i>(i: &mut &'i str) -> ModalResult<Option<&'i str>> {
|
||||
/// opt(alpha1).parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("abcd;"), Ok((";", Some("abcd"))));
|
||||
/// assert_eq!(parser.parse_peek("123;"), Ok(("123;", None)));
|
||||
/// # }
|
||||
/// ```
|
||||
pub fn opt<Input: Stream, Output, Error, ParseNext>(
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Option<Output>, Error>
|
||||
where
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
trace("opt", move |input: &mut Input| {
|
||||
let start = input.checkpoint();
|
||||
match parser.parse_next(input) {
|
||||
Ok(o) => Ok(Some(o)),
|
||||
Err(e) if e.is_backtrack() => {
|
||||
input.reset(&start);
|
||||
Ok(None)
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Calls the parser if the condition is met.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::combinator::opt;
|
||||
/// use winnow::combinator::cond;
|
||||
/// use winnow::ascii::alpha1;
|
||||
/// # fn main() {
|
||||
///
|
||||
/// fn parser<'i>(i: &mut &'i str) -> ModalResult<Option<&'i str>> {
|
||||
/// let prefix = opt("-").parse_next(i)?;
|
||||
/// let condition = prefix.is_some();
|
||||
/// cond(condition, alpha1).parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("-abcd;"), Ok((";", Some("abcd"))));
|
||||
/// assert_eq!(parser.parse_peek("abcd;"), Ok(("abcd;", None)));
|
||||
/// assert!(parser.parse_peek("-123;").is_err());
|
||||
/// assert_eq!(parser.parse_peek("123;"), Ok(("123;", None)));
|
||||
/// # }
|
||||
/// ```
|
||||
pub fn cond<Input, Output, Error, ParseNext>(
|
||||
cond: bool,
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Option<Output>, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
trace("cond", move |input: &mut Input| {
|
||||
if cond {
|
||||
parser.parse_next(input).map(Some)
|
||||
} else {
|
||||
Ok(None)
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Apply the parser without advancing the input.
|
||||
///
|
||||
/// To lookahead and only advance on success, see [`opt`].
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::peek;
|
||||
/// use winnow::ascii::alpha1;
|
||||
/// # fn main() {
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// peek(alpha1).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("abcd;"), Ok(("abcd;", "abcd")));
|
||||
/// assert!(parser.parse_peek("123;").is_err());
|
||||
/// # }
|
||||
/// ```
|
||||
#[doc(alias = "look_ahead")]
|
||||
#[doc(alias = "rewind")]
|
||||
pub fn peek<Input, Output, Error, ParseNext>(
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
{
|
||||
trace("peek", move |input: &mut Input| {
|
||||
let start = input.checkpoint();
|
||||
let res = parser.parse_next(input);
|
||||
input.reset(&start);
|
||||
res
|
||||
})
|
||||
}
|
||||
|
||||
/// Match the end of the [`Stream`]
|
||||
///
|
||||
/// Otherwise, it will error.
|
||||
///
|
||||
/// # Effective Signature
|
||||
///
|
||||
/// Assuming you are parsing a `&str` [Stream]:
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;;
|
||||
/// pub fn eof<'i>(input: &mut &'i str) -> ModalResult<&'i str>
|
||||
/// # {
|
||||
/// # winnow::combinator::eof.parse_next(input)
|
||||
/// # }
|
||||
/// ```
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use std::str;
|
||||
/// # use winnow::combinator::eof;
|
||||
/// # use winnow::prelude::*;
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// eof.parse_next(input)
|
||||
/// }
|
||||
/// assert!(parser.parse_peek("abc").is_err());
|
||||
/// assert_eq!(parser.parse_peek(""), Ok(("", "")));
|
||||
/// ```
|
||||
#[doc(alias = "end")]
|
||||
#[doc(alias = "eoi")]
|
||||
pub fn eof<Input, Error>(input: &mut Input) -> Result<<Input as Stream>::Slice, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
trace("eof", move |input: &mut Input| {
|
||||
if input.eof_offset() == 0 {
|
||||
Ok(input.next_slice(0))
|
||||
} else {
|
||||
Err(ParserError::from_input(input))
|
||||
}
|
||||
})
|
||||
.parse_next(input)
|
||||
}
|
||||
|
||||
/// Succeeds if the child parser returns an error.
|
||||
///
|
||||
/// <div class="warning">
|
||||
///
|
||||
/// **Note:** This does not advance the [`Stream`]
|
||||
///
|
||||
/// </div>
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::not;
|
||||
/// use winnow::ascii::alpha1;
|
||||
/// # fn main() {
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<()> {
|
||||
/// not(alpha1).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("123"), Ok(("123", ())));
|
||||
/// assert!(parser.parse_peek("abcd").is_err());
|
||||
/// # }
|
||||
/// ```
|
||||
pub fn not<Input, Output, Error, ParseNext>(mut parser: ParseNext) -> impl Parser<Input, (), Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
{
|
||||
trace("not", move |input: &mut Input| {
|
||||
let start = input.checkpoint();
|
||||
let res = parser.parse_next(input);
|
||||
input.reset(&start);
|
||||
match res {
|
||||
Ok(_) => Err(ParserError::from_input(input)),
|
||||
Err(e) if e.is_backtrack() => Ok(()),
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/// Transforms an [`ErrMode::Backtrack`][crate::error::ErrMode::Backtrack] (recoverable) to [`ErrMode::Cut`][crate::error::ErrMode::Cut] (unrecoverable)
|
||||
///
|
||||
/// This commits the parse result, preventing alternative branch paths like with
|
||||
/// [`winnow::combinator::alt`][crate::combinator::alt].
|
||||
///
|
||||
/// See the [tutorial][crate::_tutorial::chapter_7] for more details.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// Without `cut_err`:
|
||||
/// ```rust
|
||||
/// # use winnow::token::one_of;
|
||||
/// # use winnow::token::rest;
|
||||
/// # use winnow::ascii::digit1;
|
||||
/// # use winnow::combinator::alt;
|
||||
/// # use winnow::combinator::preceded;
|
||||
/// # use winnow::prelude::*;
|
||||
/// # fn main() {
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// alt((
|
||||
/// preceded(one_of(['+', '-']), digit1),
|
||||
/// rest
|
||||
/// )).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("+10 ab"), Ok((" ab", "10")));
|
||||
/// assert_eq!(parser.parse_peek("ab"), Ok(("", "ab")));
|
||||
/// assert_eq!(parser.parse_peek("+"), Ok(("", "+")));
|
||||
/// # }
|
||||
/// ```
|
||||
///
|
||||
/// With `cut_err`:
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::ContextError};
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::token::one_of;
|
||||
/// # use winnow::token::rest;
|
||||
/// # use winnow::ascii::digit1;
|
||||
/// # use winnow::combinator::alt;
|
||||
/// # use winnow::combinator::preceded;
|
||||
/// use winnow::combinator::cut_err;
|
||||
/// # fn main() {
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// alt((
|
||||
/// preceded(one_of(['+', '-']), cut_err(digit1)),
|
||||
/// rest
|
||||
/// )).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("+10 ab"), Ok((" ab", "10")));
|
||||
/// assert_eq!(parser.parse_peek("ab"), Ok(("", "ab")));
|
||||
/// assert_eq!(parser.parse_peek("+"), Err(ErrMode::Cut(ContextError::new())));
|
||||
/// # }
|
||||
/// ```
|
||||
pub fn cut_err<Input, Output, Error, ParseNext>(
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input> + ModalError,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
{
|
||||
trace("cut_err", move |input: &mut Input| {
|
||||
parser.parse_next(input).map_err(|e| e.cut())
|
||||
})
|
||||
}
|
||||
|
||||
/// Transforms an [`ErrMode::Cut`][crate::error::ErrMode::Cut] (unrecoverable) to [`ErrMode::Backtrack`][crate::error::ErrMode::Backtrack] (recoverable)
|
||||
///
|
||||
/// This attempts the parse, allowing other parsers to be tried on failure, like with
|
||||
/// [`winnow::combinator::alt`][crate::combinator::alt].
|
||||
pub fn backtrack_err<Input, Output, Error, ParseNext>(
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input> + ModalError,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
{
|
||||
trace("backtrack_err", move |input: &mut Input| {
|
||||
parser.parse_next(input).map_err(|e| e.backtrack())
|
||||
})
|
||||
}
|
||||
|
||||
/// A placeholder for a not-yet-implemented [`Parser`]
|
||||
///
|
||||
/// This is analogous to the [`todo!`] macro and helps with prototyping.
|
||||
///
|
||||
/// # Panic
|
||||
///
|
||||
/// This will panic when parsing
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::combinator::todo;
|
||||
///
|
||||
/// fn parser(input: &mut &str) -> ModalResult<u64> {
|
||||
/// todo(input)
|
||||
/// }
|
||||
/// ```
|
||||
#[track_caller]
|
||||
pub fn todo<Input, Output, Error>(input: &mut Input) -> Result<Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
#![allow(clippy::todo)]
|
||||
trace("todo", move |_input: &mut Input| {
|
||||
todo!("unimplemented parse")
|
||||
})
|
||||
.parse_next(input)
|
||||
}
|
||||
|
||||
/// Repeats the embedded parser, lazily returning the results
|
||||
///
|
||||
/// Call the iterator's [`ParserIterator::finish`] method to get the remaining input if successful,
|
||||
/// or the error value if we encountered an error.
|
||||
///
|
||||
/// On [`ErrMode::Backtrack`][crate::error::ErrMode::Backtrack], iteration will stop. To instead chain an error up, see [`cut_err`].
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::{combinator::iterator, ascii::alpha1, combinator::terminated};
|
||||
/// use std::collections::HashMap;
|
||||
///
|
||||
/// let data = "abc|defg|hijkl|mnopqr|123";
|
||||
/// let mut it = iterator(data, terminated(alpha1, "|"));
|
||||
///
|
||||
/// let parsed = it.map(|v| (v, v.len())).collect::<HashMap<_,_>>();
|
||||
/// let res: ModalResult<_> = it.finish();
|
||||
///
|
||||
/// assert_eq!(parsed, [("abc", 3usize), ("defg", 4), ("hijkl", 5), ("mnopqr", 6)].iter().cloned().collect());
|
||||
/// assert_eq!(res, Ok(("123", ())));
|
||||
/// ```
|
||||
pub fn iterator<Input, Output, Error, ParseNext>(
|
||||
input: Input,
|
||||
parser: ParseNext,
|
||||
) -> ParserIterator<ParseNext, Input, Output, Error>
|
||||
where
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
ParserIterator {
|
||||
parser,
|
||||
input,
|
||||
state: State::Running,
|
||||
o: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Main structure associated to [`iterator`].
|
||||
pub struct ParserIterator<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
{
|
||||
parser: F,
|
||||
input: I,
|
||||
state: State<E>,
|
||||
o: core::marker::PhantomData<O>,
|
||||
}
|
||||
|
||||
impl<F, I, O, E> ParserIterator<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
/// Returns the remaining input if parsing was successful, or the error if we encountered an error.
|
||||
pub fn finish(self) -> Result<(I, ()), E> {
|
||||
match self.state {
|
||||
State::Running | State::Done => Ok((self.input, ())),
|
||||
State::Cut(e) => Err(e),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<F, I, O, E> core::iter::Iterator for &mut ParserIterator<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
type Item = O;
|
||||
|
||||
fn next(&mut self) -> Option<Self::Item> {
|
||||
if matches!(self.state, State::Running) {
|
||||
let start = self.input.checkpoint();
|
||||
|
||||
match self.parser.parse_next(&mut self.input) {
|
||||
Ok(o) => {
|
||||
self.state = State::Running;
|
||||
Some(o)
|
||||
}
|
||||
Err(e) if e.is_backtrack() => {
|
||||
self.input.reset(&start);
|
||||
self.state = State::Done;
|
||||
None
|
||||
}
|
||||
Err(e) => {
|
||||
self.state = State::Cut(e);
|
||||
None
|
||||
}
|
||||
}
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
enum State<E> {
|
||||
Running,
|
||||
Done,
|
||||
Cut(E),
|
||||
}
|
||||
|
||||
/// Succeed, consuming no input
|
||||
///
|
||||
/// For example, it can be used as the last alternative in `alt` to
|
||||
/// specify the default case.
|
||||
///
|
||||
/// Useful with:
|
||||
/// - [`Parser::value`]
|
||||
/// - [`Parser::default_value`]
|
||||
/// - [`Parser::map`]
|
||||
///
|
||||
/// <div class="warning">
|
||||
///
|
||||
/// **Note:** This never advances the [`Stream`]
|
||||
///
|
||||
/// </div>
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::alt;
|
||||
/// use winnow::combinator::empty;
|
||||
///
|
||||
/// fn sign(input: &mut &str) -> ModalResult<isize> {
|
||||
/// alt((
|
||||
/// '-'.value(-1),
|
||||
/// '+'.value(1),
|
||||
/// empty.value(1)
|
||||
/// )).parse_next(input)
|
||||
/// }
|
||||
/// assert_eq!(sign.parse_peek("+10"), Ok(("10", 1)));
|
||||
/// assert_eq!(sign.parse_peek("-10"), Ok(("10", -1)));
|
||||
/// assert_eq!(sign.parse_peek("10"), Ok(("10", 1)));
|
||||
/// ```
|
||||
#[doc(alias = "value")]
|
||||
#[doc(alias = "success")]
|
||||
#[inline]
|
||||
pub fn empty<Input, Error>(_input: &mut Input) -> Result<(), Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// A parser which always fails.
|
||||
///
|
||||
/// For example, it can be used as the last alternative in `alt` to
|
||||
/// control the error message given.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::InputError};
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::fail;
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<(), InputError<&'i str>> {
|
||||
/// fail.parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("string"), Err(ErrMode::Backtrack(InputError::at("string"))));
|
||||
/// ```
|
||||
#[doc(alias = "unexpected")]
|
||||
#[inline]
|
||||
pub fn fail<Input, Output, Error>(i: &mut Input) -> Result<Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
{
|
||||
trace("fail", |i: &mut Input| Err(ParserError::from_input(i))).parse_next(i)
|
||||
}
|
||||
@@ -0,0 +1,304 @@
|
||||
#![cfg(feature = "std")]
|
||||
|
||||
use std::io::Write;
|
||||
|
||||
use crate::error::ParserError;
|
||||
use crate::stream::Stream;
|
||||
use crate::*;
|
||||
|
||||
pub(crate) struct Trace<P, D, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
D: std::fmt::Display,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
parser: P,
|
||||
name: D,
|
||||
call_count: usize,
|
||||
i: core::marker::PhantomData<I>,
|
||||
o: core::marker::PhantomData<O>,
|
||||
e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<P, D, I, O, E> Trace<P, D, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
D: std::fmt::Display,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline(always)]
|
||||
pub(crate) fn new(parser: P, name: D) -> Self {
|
||||
Self {
|
||||
parser,
|
||||
name,
|
||||
call_count: 0,
|
||||
i: Default::default(),
|
||||
o: Default::default(),
|
||||
e: Default::default(),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<P, D, I, O, E> Parser<I, O, E> for Trace<P, D, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
D: std::fmt::Display,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E> {
|
||||
let depth = Depth::new();
|
||||
let original = i.checkpoint();
|
||||
start(*depth, &self.name, self.call_count, i);
|
||||
|
||||
let res = self.parser.parse_next(i);
|
||||
|
||||
let consumed = i.offset_from(&original);
|
||||
let severity = Severity::with_result(&res);
|
||||
end(*depth, &self.name, self.call_count, consumed, severity);
|
||||
self.call_count += 1;
|
||||
|
||||
res
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct Depth {
|
||||
depth: usize,
|
||||
inc: bool,
|
||||
}
|
||||
|
||||
impl Depth {
|
||||
pub(crate) fn new() -> Self {
|
||||
let depth = DEPTH.fetch_add(1, std::sync::atomic::Ordering::SeqCst);
|
||||
let inc = true;
|
||||
Self { depth, inc }
|
||||
}
|
||||
|
||||
pub(crate) fn existing() -> Self {
|
||||
let depth = DEPTH.load(std::sync::atomic::Ordering::SeqCst);
|
||||
let inc = false;
|
||||
Self { depth, inc }
|
||||
}
|
||||
}
|
||||
|
||||
impl Drop for Depth {
|
||||
fn drop(&mut self) {
|
||||
if self.inc {
|
||||
let _ = DEPTH.fetch_sub(1, std::sync::atomic::Ordering::SeqCst);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<usize> for Depth {
|
||||
#[inline(always)]
|
||||
fn as_ref(&self) -> &usize {
|
||||
&self.depth
|
||||
}
|
||||
}
|
||||
|
||||
impl core::ops::Deref for Depth {
|
||||
type Target = usize;
|
||||
|
||||
#[inline(always)]
|
||||
fn deref(&self) -> &Self::Target {
|
||||
&self.depth
|
||||
}
|
||||
}
|
||||
|
||||
static DEPTH: std::sync::atomic::AtomicUsize = std::sync::atomic::AtomicUsize::new(0);
|
||||
|
||||
pub(crate) enum Severity {
|
||||
Success,
|
||||
Backtrack,
|
||||
Cut,
|
||||
Incomplete,
|
||||
}
|
||||
|
||||
impl Severity {
|
||||
pub(crate) fn with_result<T, I: Stream, E: ParserError<I>>(result: &Result<T, E>) -> Self {
|
||||
match result {
|
||||
Ok(_) => Self::Success,
|
||||
Err(e) if e.is_backtrack() => Self::Backtrack,
|
||||
Err(e) if e.is_incomplete() => Self::Incomplete,
|
||||
_ => Self::Cut,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) fn start<I: Stream>(
|
||||
depth: usize,
|
||||
name: &dyn core::fmt::Display,
|
||||
count: usize,
|
||||
input: &I,
|
||||
) {
|
||||
let gutter_style = anstyle::Style::new().bold();
|
||||
let input_style = anstyle::Style::new().underline();
|
||||
let eof_style = anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Cyan.into()));
|
||||
|
||||
let (call_width, input_width) = column_widths();
|
||||
|
||||
let count = if 0 < count {
|
||||
format!(":{count}")
|
||||
} else {
|
||||
"".to_owned()
|
||||
};
|
||||
let call_column = format!("{:depth$}> {name}{count}", "");
|
||||
|
||||
// The debug version of `slice` might be wider, either due to rendering one byte as two nibbles or
|
||||
// escaping in strings.
|
||||
let mut debug_slice = format!("{:?}", crate::util::from_fn(|f| input.trace(f)));
|
||||
let (debug_slice, eof) = if let Some(debug_offset) = debug_slice
|
||||
.char_indices()
|
||||
.enumerate()
|
||||
.find_map(|(pos, (offset, _))| (input_width <= pos).then_some(offset))
|
||||
{
|
||||
debug_slice.truncate(debug_offset);
|
||||
let eof = "";
|
||||
(debug_slice, eof)
|
||||
} else {
|
||||
let eof = if debug_slice.chars().count() < input_width {
|
||||
"∅"
|
||||
} else {
|
||||
""
|
||||
};
|
||||
(debug_slice, eof)
|
||||
};
|
||||
|
||||
let writer = anstream::stderr();
|
||||
let mut writer = writer.lock();
|
||||
let _ = writeln!(
|
||||
writer,
|
||||
"{call_column:call_width$} {gutter_style}|{gutter_reset} {input_style}{debug_slice}{input_reset}{eof_style}{eof}{eof_reset}",
|
||||
gutter_style=gutter_style.render(),
|
||||
gutter_reset=gutter_style.render_reset(),
|
||||
input_style=input_style.render(),
|
||||
input_reset=input_style.render_reset(),
|
||||
eof_style=eof_style.render(),
|
||||
eof_reset=eof_style.render_reset(),
|
||||
);
|
||||
}
|
||||
|
||||
pub(crate) fn end(
|
||||
depth: usize,
|
||||
name: &dyn core::fmt::Display,
|
||||
count: usize,
|
||||
consumed: usize,
|
||||
severity: Severity,
|
||||
) {
|
||||
let gutter_style = anstyle::Style::new().bold();
|
||||
|
||||
let (call_width, _) = column_widths();
|
||||
|
||||
let count = if 0 < count {
|
||||
format!(":{count}")
|
||||
} else {
|
||||
"".to_owned()
|
||||
};
|
||||
let call_column = format!("{:depth$}< {name}{count}", "");
|
||||
|
||||
let (status_style, status) = match severity {
|
||||
Severity::Success => {
|
||||
let style = anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Green.into()));
|
||||
let status = format!("+{consumed}");
|
||||
(style, status)
|
||||
}
|
||||
Severity::Backtrack => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Yellow.into())),
|
||||
"backtrack".to_owned(),
|
||||
),
|
||||
Severity::Cut => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Red.into())),
|
||||
"cut".to_owned(),
|
||||
),
|
||||
Severity::Incomplete => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Red.into())),
|
||||
"incomplete".to_owned(),
|
||||
),
|
||||
};
|
||||
|
||||
let writer = anstream::stderr();
|
||||
let mut writer = writer.lock();
|
||||
let _ = writeln!(
|
||||
writer,
|
||||
"{status_style}{call_column:call_width$}{status_reset} {gutter_style}|{gutter_reset} {status_style}{status}{status_reset}",
|
||||
gutter_style=gutter_style.render(),
|
||||
gutter_reset=gutter_style.render_reset(),
|
||||
status_style=status_style.render(),
|
||||
status_reset=status_style.render_reset(),
|
||||
);
|
||||
}
|
||||
|
||||
pub(crate) fn result(depth: usize, name: &dyn core::fmt::Display, severity: Severity) {
|
||||
let gutter_style = anstyle::Style::new().bold();
|
||||
|
||||
let (call_width, _) = column_widths();
|
||||
|
||||
let call_column = format!("{:depth$}| {name}", "");
|
||||
|
||||
let (status_style, status) = match severity {
|
||||
Severity::Success => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Green.into())),
|
||||
"",
|
||||
),
|
||||
Severity::Backtrack => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Yellow.into())),
|
||||
"backtrack",
|
||||
),
|
||||
Severity::Cut => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Red.into())),
|
||||
"cut",
|
||||
),
|
||||
Severity::Incomplete => (
|
||||
anstyle::Style::new().fg_color(Some(anstyle::AnsiColor::Red.into())),
|
||||
"incomplete",
|
||||
),
|
||||
};
|
||||
|
||||
let writer = anstream::stderr();
|
||||
let mut writer = writer.lock();
|
||||
let _ = writeln!(
|
||||
writer,
|
||||
"{status_style}{call_column:call_width$}{status_reset} {gutter_style}|{gutter_reset} {status_style}{status}{status_reset}",
|
||||
gutter_style=gutter_style.render(),
|
||||
gutter_reset=gutter_style.render_reset(),
|
||||
status_style=status_style.render(),
|
||||
status_reset=status_style.render_reset(),
|
||||
);
|
||||
}
|
||||
|
||||
fn column_widths() -> (usize, usize) {
|
||||
let term_width = term_width();
|
||||
|
||||
let min_call_width = 40;
|
||||
let min_input_width = 20;
|
||||
let decor_width = 3;
|
||||
let extra_width = term_width
|
||||
.checked_sub(min_call_width + min_input_width + decor_width)
|
||||
.unwrap_or_default();
|
||||
let call_width = min_call_width + 2 * extra_width / 3;
|
||||
let input_width = min_input_width + extra_width / 3;
|
||||
|
||||
(call_width, input_width)
|
||||
}
|
||||
|
||||
fn term_width() -> usize {
|
||||
columns_env().or_else(query_width).unwrap_or(80)
|
||||
}
|
||||
|
||||
fn query_width() -> Option<usize> {
|
||||
use is_terminal_polyfill::IsTerminal;
|
||||
if std::io::stderr().is_terminal() {
|
||||
terminal_size::terminal_size().map(|(w, _h)| w.0.into())
|
||||
} else {
|
||||
None
|
||||
}
|
||||
}
|
||||
|
||||
fn columns_env() -> Option<usize> {
|
||||
std::env::var("COLUMNS")
|
||||
.ok()
|
||||
.and_then(|c| c.parse::<usize>().ok())
|
||||
}
|
||||
@@ -0,0 +1,96 @@
|
||||
#![cfg_attr(feature = "debug", allow(clippy::std_instead_of_core))]
|
||||
|
||||
#[cfg(feature = "debug")]
|
||||
mod internals;
|
||||
|
||||
use crate::error::ParserError;
|
||||
use crate::stream::Stream;
|
||||
use crate::Parser;
|
||||
|
||||
/// Trace the execution of the parser
|
||||
///
|
||||
/// Note that [`Parser::context`] also provides high level trace information.
|
||||
///
|
||||
/// See [tutorial][crate::_tutorial::chapter_8] for more details.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::token::take_while;
|
||||
/// # use winnow::stream::AsChar;
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::trace;
|
||||
///
|
||||
/// fn short_alpha<'s>(s: &mut &'s [u8]) -> ModalResult<&'s [u8]> {
|
||||
/// trace("short_alpha",
|
||||
/// take_while(3..=6, AsChar::is_alpha)
|
||||
/// ).parse_next(s)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(short_alpha.parse_peek(b"latin123"), Ok((&b"123"[..], &b"latin"[..])));
|
||||
/// assert_eq!(short_alpha.parse_peek(b"lengthy"), Ok((&b"y"[..], &b"length"[..])));
|
||||
/// assert_eq!(short_alpha.parse_peek(b"latin"), Ok((&b""[..], &b"latin"[..])));
|
||||
/// assert!(short_alpha.parse_peek(b"ed").is_err());
|
||||
/// assert!(short_alpha.parse_peek(b"12345").is_err());
|
||||
/// ```
|
||||
#[cfg_attr(not(feature = "debug"), allow(unused_variables))]
|
||||
#[cfg_attr(not(feature = "debug"), allow(unused_mut))]
|
||||
#[cfg_attr(not(feature = "debug"), inline(always))]
|
||||
pub fn trace<I: Stream, O, E: ParserError<I>>(
|
||||
name: impl core::fmt::Display,
|
||||
parser: impl Parser<I, O, E>,
|
||||
) -> impl Parser<I, O, E> {
|
||||
#[cfg(feature = "debug")]
|
||||
{
|
||||
internals::Trace::new(parser, name)
|
||||
}
|
||||
#[cfg(not(feature = "debug"))]
|
||||
{
|
||||
parser
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg_attr(not(feature = "debug"), allow(unused_variables))]
|
||||
pub(crate) fn trace_result<T, I: Stream, E: ParserError<I>>(
|
||||
name: impl core::fmt::Display,
|
||||
res: &Result<T, E>,
|
||||
) {
|
||||
#[cfg(feature = "debug")]
|
||||
{
|
||||
let depth = internals::Depth::existing();
|
||||
let severity = internals::Severity::with_result(res);
|
||||
internals::result(*depth, &name, severity);
|
||||
}
|
||||
}
|
||||
|
||||
pub(crate) struct DisplayDebug<D>(pub(crate) D);
|
||||
|
||||
impl<D: core::fmt::Debug> core::fmt::Display for DisplayDebug<D> {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
write!(f, "{:?}", self.0)
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg(feature = "std")]
|
||||
#[cfg_attr(miri, ignore)]
|
||||
#[cfg(unix)]
|
||||
#[cfg(feature = "debug")]
|
||||
fn example() {
|
||||
use term_transcript::{test::TestConfig, ShellOptions};
|
||||
|
||||
let path = snapbox::cmd::compile_example("string", ["--features=debug"]).unwrap();
|
||||
|
||||
let current_dir = path.parent().unwrap();
|
||||
let cmd = path.file_name().unwrap();
|
||||
// HACK: term_transcript doesn't allow non-UTF8 paths
|
||||
let cmd = format!("./{}", cmd.to_string_lossy());
|
||||
|
||||
TestConfig::new(
|
||||
ShellOptions::default()
|
||||
.with_current_dir(current_dir)
|
||||
.with_env("CLICOLOR_FORCE", "1"),
|
||||
)
|
||||
.test("assets/trace.svg", [format!(r#"{cmd} '"abc"'"#).as_str()]);
|
||||
}
|
||||
+569
@@ -0,0 +1,569 @@
|
||||
use crate::combinator::empty;
|
||||
use crate::combinator::fail;
|
||||
use crate::combinator::opt;
|
||||
use crate::combinator::trace;
|
||||
use crate::error::ParserError;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::Parser;
|
||||
use crate::Result;
|
||||
|
||||
/// Parses an expression based on operator precedence.
|
||||
///
|
||||
/// It uses a Pratt parsing algorithm, where operators are
|
||||
/// associated with a binding power. The higher the power,
|
||||
/// the more tightly an operator will bind to its operands.
|
||||
///
|
||||
/// This method returns an [`Expression`], which configures
|
||||
/// the Pratt parser.
|
||||
///
|
||||
/// Each operator type is configured with [`Prefix`], [`Postfix`],
|
||||
/// and [`Infix`]. These describe the operator's binding power,
|
||||
/// a function that applies the operator to its operand, and the
|
||||
/// operator's associativity (infix only).
|
||||
///
|
||||
/// For a more full-featured example, look at the [C-style Expression][crate::_topic::arithmetic#c-style-expression]
|
||||
/// topic.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// Parsing a simple arithmetic expression without parenthesis.
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// # use winnow::ascii::digit1;
|
||||
/// # use winnow::combinator::{dispatch, fail};
|
||||
/// # use winnow::token::any;
|
||||
/// use winnow::combinator::expression;
|
||||
/// use winnow::combinator::{Prefix, Postfix, Infix};
|
||||
///
|
||||
/// fn parser<'i>() -> impl Parser<&'i str, i32, ContextError> {
|
||||
/// move |i: &mut &str| {
|
||||
/// use Infix::*;
|
||||
/// expression(digit1.parse_to::<i32>()) // operands are 32-bit integers
|
||||
/// .prefix(dispatch! {any;
|
||||
/// '-' => Prefix(12, |_, a: i32| Ok(-a)),
|
||||
/// _ => fail,
|
||||
/// })
|
||||
/// .infix(dispatch! {any;
|
||||
/// '+' => Left(5, |_, a, b| Ok(a + b)),
|
||||
/// '-' => Left(5, |_, a, b| Ok(a - b)),
|
||||
/// '*' => Left(7, |_, a, b| Ok(a * b)),
|
||||
/// '/' => Left(7, |_, a: i32, b| Ok(a.checked_div(b).unwrap_or_default())),
|
||||
/// _ => fail,
|
||||
/// })
|
||||
/// .postfix(dispatch! {any;
|
||||
/// '!' => Postfix(15, |_, a| if a < 1 { Ok(1) } else { Ok((1..=a).fold(1, |acc, a| acc*a)) }),
|
||||
/// _ => fail,
|
||||
/// })
|
||||
/// .parse_next(i)
|
||||
/// }
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser().parse("1+1"), Ok(2));
|
||||
/// assert_eq!(parser().parse("0!"), Ok(1));
|
||||
/// assert_eq!(parser().parse("-1*5*2*10+30/3!"), Ok(-95));
|
||||
/// ```
|
||||
#[doc(alias = "pratt")]
|
||||
#[doc(alias = "separated")]
|
||||
#[doc(alias = "shunting_yard")]
|
||||
#[doc(alias = "precedence_climbing")]
|
||||
#[inline(always)]
|
||||
#[allow(clippy::type_complexity)]
|
||||
pub fn expression<I, ParseOperand, O, E>(
|
||||
parse_operand: ParseOperand,
|
||||
) -> Expression<
|
||||
I,
|
||||
O,
|
||||
ParseOperand,
|
||||
impl Parser<I, Prefix<I, O, E>, E>,
|
||||
impl Parser<I, Postfix<I, O, E>, E>,
|
||||
impl Parser<I, Infix<I, O, E>, E>,
|
||||
E,
|
||||
>
|
||||
where
|
||||
I: Stream + StreamIsPartial,
|
||||
ParseOperand: Parser<I, O, E>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
Expression {
|
||||
precedence_level: 0,
|
||||
parse_operand,
|
||||
parse_prefix: fail,
|
||||
parse_postfix: fail,
|
||||
parse_infix: fail,
|
||||
i: Default::default(),
|
||||
o: Default::default(),
|
||||
e: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// A helper struct for [`expression()`].
|
||||
///
|
||||
/// Holds the configuration for the Pratt parser, including
|
||||
/// the operator and operand parsers. A precedence level can
|
||||
/// also be set, which is useful to disambiguate parse trees
|
||||
/// based on the parent operator's precedence.
|
||||
///
|
||||
/// Implements [`Parser`]. When parsing an input, it applies
|
||||
/// the Pratt parser.
|
||||
pub struct Expression<I, O, ParseOperand, Pre, Post, Pix, E>
|
||||
where
|
||||
I: Stream + StreamIsPartial,
|
||||
ParseOperand: Parser<I, O, E>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
precedence_level: i64,
|
||||
parse_operand: ParseOperand,
|
||||
parse_prefix: Pre,
|
||||
parse_postfix: Post,
|
||||
parse_infix: Pix,
|
||||
i: core::marker::PhantomData<I>,
|
||||
o: core::marker::PhantomData<O>,
|
||||
e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<I, O, ParseOperand, Pre, Post, Pix, E> Expression<I, O, ParseOperand, Pre, Post, Pix, E>
|
||||
where
|
||||
ParseOperand: Parser<I, O, E>,
|
||||
I: Stream + StreamIsPartial,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
/// Sets the prefix operator parser.
|
||||
///
|
||||
/// The parser should parse the input to a [`Prefix`],
|
||||
/// which contains the operator's binding power and
|
||||
/// a fold function which applies the operator to its
|
||||
/// operands.
|
||||
#[inline(always)]
|
||||
pub fn prefix<NewParsePrefix>(
|
||||
self,
|
||||
parser: NewParsePrefix,
|
||||
) -> Expression<I, O, ParseOperand, NewParsePrefix, Post, Pix, E>
|
||||
where
|
||||
NewParsePrefix: Parser<I, Prefix<I, O, E>, E>,
|
||||
{
|
||||
Expression {
|
||||
precedence_level: self.precedence_level,
|
||||
parse_operand: self.parse_operand,
|
||||
parse_prefix: parser,
|
||||
parse_postfix: self.parse_postfix,
|
||||
parse_infix: self.parse_infix,
|
||||
i: Default::default(),
|
||||
o: Default::default(),
|
||||
e: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Sets the postfix operator parser.
|
||||
///
|
||||
/// The parser should parse the input to a [`Postfix`],
|
||||
/// which contains the operator's binding power and
|
||||
/// a fold function which applies the operator to its
|
||||
/// operands.
|
||||
#[inline(always)]
|
||||
pub fn postfix<NewParsePostfix>(
|
||||
self,
|
||||
parser: NewParsePostfix,
|
||||
) -> Expression<I, O, ParseOperand, Pre, NewParsePostfix, Pix, E>
|
||||
where
|
||||
NewParsePostfix: Parser<I, Postfix<I, O, E>, E>,
|
||||
{
|
||||
Expression {
|
||||
precedence_level: self.precedence_level,
|
||||
parse_operand: self.parse_operand,
|
||||
parse_prefix: self.parse_prefix,
|
||||
parse_postfix: parser,
|
||||
parse_infix: self.parse_infix,
|
||||
i: Default::default(),
|
||||
o: Default::default(),
|
||||
e: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Sets the infix operator parser.
|
||||
///
|
||||
/// The parser should parse the input to a [`Infix`],
|
||||
/// which contains the operator's binding power and
|
||||
/// a fold function which applies the operator to its
|
||||
/// operands.
|
||||
#[inline(always)]
|
||||
pub fn infix<NewParseInfix>(
|
||||
self,
|
||||
parser: NewParseInfix,
|
||||
) -> Expression<I, O, ParseOperand, Pre, Post, NewParseInfix, E>
|
||||
where
|
||||
NewParseInfix: Parser<I, Infix<I, O, E>, E>,
|
||||
{
|
||||
Expression {
|
||||
precedence_level: self.precedence_level,
|
||||
parse_operand: self.parse_operand,
|
||||
parse_prefix: self.parse_prefix,
|
||||
parse_postfix: self.parse_postfix,
|
||||
parse_infix: parser,
|
||||
i: Default::default(),
|
||||
o: Default::default(),
|
||||
e: Default::default(),
|
||||
}
|
||||
}
|
||||
|
||||
/// Sets the precedence level for the current instance of the parser.
|
||||
///
|
||||
/// It defaults to 0, which is traditionally treated as the "lowest"
|
||||
/// possible precedence when parsing an expression.
|
||||
///
|
||||
/// This is useful to disambiguate grammars based on the parent operator's
|
||||
/// precedence. This comes up primarily when parsing recursive expressions.
|
||||
///
|
||||
/// The parsing machinery underpinning [`Expression`] assumes that a "more
|
||||
/// tightly binding" operator is numerically large, while a "more loosely
|
||||
/// binding" operator is numerically small. For example, `13` is a higher
|
||||
/// precedence level than `1` because `13 > 1`.
|
||||
///
|
||||
/// Other ways of describing this relationship:
|
||||
/// - `13` has a higher precedence compared to `1`
|
||||
/// - `13` has a higher binding power compared to `1`
|
||||
///
|
||||
/// Note: Binding power and precedence both refer to the same concept and
|
||||
/// may be used interchangeably.
|
||||
///
|
||||
/// # Motivation
|
||||
///
|
||||
/// If you don't understand why this is useful to have, this section tries
|
||||
/// to explain in more detail.
|
||||
///
|
||||
/// The [C-style Expressions][crate::_topic::arithmetic#c-style-expression]
|
||||
/// example has source code for parsing the expression described below, and
|
||||
/// can provide a clearer usage example.
|
||||
///
|
||||
/// Consider the following expression in the C language:
|
||||
///
|
||||
/// ```c
|
||||
/// int x = (1 == 1 ? 0 : 1, -123); // <-- let's parse this
|
||||
/// printf("%d\n", x); // -123
|
||||
/// ```
|
||||
///
|
||||
/// Let's look at the right-hand side of the expression on the first line,
|
||||
/// and replace some of the sub-expressions with symbols:
|
||||
///
|
||||
/// ```text
|
||||
/// (1 == 1 ? 0 : 1, -123) // rhs
|
||||
/// (a ? b : c, d ) // symbolic
|
||||
/// (a ? b : c, d) // remove whitespace
|
||||
/// (, (? a b c) d) // prefix notation
|
||||
/// ```
|
||||
///
|
||||
/// Written symbolically:
|
||||
/// - `a` is the condition, like `1 == 1`
|
||||
/// - `b` is the value when the condition is true
|
||||
/// - `c` is the value when the condition is false
|
||||
/// - `d` is a secondary expression unrelated to the ternary
|
||||
///
|
||||
/// In prefix notation, it's easier to see the specific operators and what
|
||||
/// they bind to:
|
||||
/// - COMMA (`,`) binds to `(? a b c)` and `d`
|
||||
/// - TERNARY (`?`) binds to `a`, `b`, and `c`
|
||||
///
|
||||
/// ## Parsing `c` and `d`
|
||||
///
|
||||
/// Let's focus on parsing the sub-expressions `c` and `d`, as that
|
||||
/// motivates why a parser precedence level is necessary.
|
||||
///
|
||||
/// To parse `c`, we would really like to re-use the parser produced by
|
||||
/// [`expression()`], because `c` is really *any* valid expression that
|
||||
/// can be parsed by `expression()` already.
|
||||
///
|
||||
/// However, we can't re-use the parser naively. When parsing `c`, we need
|
||||
/// to "escape" from the inner parser when encountering the comma separating
|
||||
/// `c` from `d`.
|
||||
///
|
||||
/// The reason we have to "escape" is because of how operator precedence is
|
||||
/// defined in the C language: the comma operator has the lowest precedence
|
||||
/// among all the operators. When we're parsing `c`, we're in the context of
|
||||
/// the ternary operator. We don't want to parse any valid expression! Just
|
||||
/// what the ternary operator captures.
|
||||
///
|
||||
/// That's where the precedence level comes in: you specify the minimum
|
||||
/// precedence this parser is willing to accept. If you come across an
|
||||
/// expression in the top-level with a lower binding power than the starting
|
||||
/// precedence, you know to stop parsing.
|
||||
///
|
||||
/// The parsing machinery inside of [`Expression`] handles most of this for
|
||||
/// you, but it can't determine what the precedence level should be for a
|
||||
/// given expression. That's a language-specific detail, and it depends on
|
||||
/// what you want to parse.
|
||||
#[inline(always)]
|
||||
pub fn current_precedence_level(
|
||||
mut self,
|
||||
level: i64,
|
||||
) -> Expression<I, O, ParseOperand, Pre, Post, Pix, E> {
|
||||
self.precedence_level = level;
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, O, Pop, Pre, Post, Pix, E> Parser<I, O, E> for Expression<I, O, Pop, Pre, Post, Pix, E>
|
||||
where
|
||||
I: Stream + StreamIsPartial,
|
||||
Pop: Parser<I, O, E>,
|
||||
Pix: Parser<I, Infix<I, O, E>, E>,
|
||||
Pre: Parser<I, Prefix<I, O, E>, E>,
|
||||
Post: Parser<I, Postfix<I, O, E>, E>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O, E> {
|
||||
trace("expression", move |i: &mut I| {
|
||||
expression_impl(
|
||||
i,
|
||||
&mut self.parse_operand,
|
||||
&mut self.parse_prefix,
|
||||
&mut self.parse_postfix,
|
||||
&mut self.parse_infix,
|
||||
self.precedence_level,
|
||||
)
|
||||
})
|
||||
.parse_next(input)
|
||||
}
|
||||
}
|
||||
|
||||
/// Opaque implementation of the Pratt parser.
|
||||
fn expression_impl<I, O, Pop, Pre, Post, Pix, E>(
|
||||
i: &mut I,
|
||||
parse_operand: &mut Pop,
|
||||
prefix: &mut Pre,
|
||||
postfix: &mut Post,
|
||||
infix: &mut Pix,
|
||||
min_power: i64,
|
||||
) -> Result<O, E>
|
||||
where
|
||||
I: Stream + StreamIsPartial,
|
||||
Pop: Parser<I, O, E>,
|
||||
Pix: Parser<I, Infix<I, O, E>, E>,
|
||||
Pre: Parser<I, Prefix<I, O, E>, E>,
|
||||
Post: Parser<I, Postfix<I, O, E>, E>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
let operand = opt(trace("operand", parse_operand.by_ref())).parse_next(i)?;
|
||||
let mut operand = if let Some(operand) = operand {
|
||||
operand
|
||||
} else {
|
||||
// Prefix unary operators
|
||||
let len = i.eof_offset();
|
||||
let Prefix(power, fold_prefix) = trace("prefix", prefix.by_ref()).parse_next(i)?;
|
||||
// infinite loop check: the parser must always consume
|
||||
if i.eof_offset() == len {
|
||||
return Err(E::assert(i, "`prefix` parsers must always consume"));
|
||||
}
|
||||
let operand = expression_impl(i, parse_operand, prefix, postfix, infix, power)?;
|
||||
fold_prefix(i, operand)?
|
||||
};
|
||||
|
||||
// A variable to stop the `'parse` loop when `Assoc::Neither` with the same
|
||||
// precedence is encountered e.g. `a == b == c`. `Assoc::Neither` has similar
|
||||
// associativity rules as `Assoc::Left`, but we stop parsing when the next operator
|
||||
// is the same as the current one.
|
||||
let mut prev_op_is_neither = None;
|
||||
'parse: while i.eof_offset() > 0 {
|
||||
// Postfix unary operators
|
||||
let start = i.checkpoint();
|
||||
if let Some(Postfix(power, fold_postfix)) =
|
||||
opt(trace("postfix", postfix.by_ref())).parse_next(i)?
|
||||
{
|
||||
// control precedence over the prefix e.g.:
|
||||
// `--(i++)` or `(--i)++`
|
||||
if power < min_power {
|
||||
i.reset(&start);
|
||||
break 'parse;
|
||||
}
|
||||
operand = fold_postfix(i, operand)?;
|
||||
|
||||
continue 'parse;
|
||||
}
|
||||
|
||||
// Infix binary operators
|
||||
let start = i.checkpoint();
|
||||
let parse_result = opt(trace("infix", infix.by_ref())).parse_next(i)?;
|
||||
if let Some(infix_op) = parse_result {
|
||||
let mut is_neither = None;
|
||||
let (lpower, rpower, fold_infix) = match infix_op {
|
||||
Infix::Right(p, f) => (p, p - 1, f),
|
||||
Infix::Left(p, f) => (p, p + 1, f),
|
||||
Infix::Neither(p, f) => {
|
||||
is_neither = Some(p);
|
||||
(p, p + 1, f)
|
||||
}
|
||||
};
|
||||
if lpower < min_power
|
||||
// MSRV: `is_some_and`
|
||||
|| match prev_op_is_neither {
|
||||
None => false,
|
||||
Some(p) => lpower == p,
|
||||
}
|
||||
{
|
||||
i.reset(&start);
|
||||
break 'parse;
|
||||
}
|
||||
prev_op_is_neither = is_neither;
|
||||
let rhs = expression_impl(i, parse_operand, prefix, postfix, infix, rpower)?;
|
||||
operand = fold_infix(i, operand, rhs)?;
|
||||
|
||||
continue 'parse;
|
||||
}
|
||||
|
||||
break 'parse;
|
||||
}
|
||||
|
||||
Ok(operand)
|
||||
}
|
||||
|
||||
/// Define an [`expression()`]'s prefix operator
|
||||
///
|
||||
/// It requires an operator binding power, as well as a
|
||||
/// fold function which applies the operator.
|
||||
pub struct Prefix<I, O, E>(
|
||||
/// Binding power
|
||||
pub i64,
|
||||
/// Unary operator
|
||||
pub fn(&mut I, O) -> Result<O, E>,
|
||||
);
|
||||
|
||||
impl<I, O, E> Clone for Prefix<I, O, E> {
|
||||
#[inline(always)]
|
||||
fn clone(&self) -> Self {
|
||||
Prefix(self.0, self.1)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream, O, E: ParserError<I>> Parser<I, Prefix<I, O, E>, E> for Prefix<I, O, E> {
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<Prefix<I, O, E>, E> {
|
||||
empty.value(self.clone()).parse_next(input)
|
||||
}
|
||||
}
|
||||
|
||||
/// Define an [`expression()`]'s postfix operator
|
||||
///
|
||||
/// It requires an operator binding power, as well as a
|
||||
/// fold function which applies the operator.
|
||||
pub struct Postfix<I, O, E>(
|
||||
/// Binding power
|
||||
pub i64,
|
||||
/// Unary operator
|
||||
pub fn(&mut I, O) -> Result<O, E>,
|
||||
);
|
||||
|
||||
impl<I, O, E> Clone for Postfix<I, O, E> {
|
||||
#[inline(always)]
|
||||
fn clone(&self) -> Self {
|
||||
Postfix(self.0, self.1)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream, O, E: ParserError<I>> Parser<I, Postfix<I, O, E>, E> for Postfix<I, O, E> {
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<Postfix<I, O, E>, E> {
|
||||
empty.value(self.clone()).parse_next(input)
|
||||
}
|
||||
}
|
||||
|
||||
/// Define an [`expression()`]'s infix operator
|
||||
///
|
||||
/// It requires an operator binding power, as well as a
|
||||
/// fold function which applies the operator.
|
||||
pub enum Infix<I, O, E> {
|
||||
/// Left-associative operator
|
||||
///
|
||||
/// The operators will bind more tightly to their rightmost operands.
|
||||
///
|
||||
/// e.g `A op B op C` -> `(A op B) op C`
|
||||
Left(
|
||||
/// Binding power
|
||||
i64,
|
||||
/// Binary operator
|
||||
fn(&mut I, O, O) -> Result<O, E>,
|
||||
),
|
||||
/// Right-associative operator
|
||||
///
|
||||
/// The operators will bind more tightly to their leftmost operands.
|
||||
///
|
||||
/// e.g `A op B op C` -> `A op (B op C)`
|
||||
Right(
|
||||
/// Binding power
|
||||
i64,
|
||||
/// Binary operator
|
||||
fn(&mut I, O, O) -> Result<O, E>,
|
||||
),
|
||||
/// Neither left or right associative
|
||||
///
|
||||
/// `Infix::Neither` has similar associativity rules as `Assoc::Left`, but we stop
|
||||
/// parsing when the next operator is the same as the current one.
|
||||
///
|
||||
/// e.g. `a == b == c` -> `(a == b)`, fail: `(== c)`
|
||||
Neither(
|
||||
/// Binding power
|
||||
i64,
|
||||
/// Binary operator
|
||||
fn(&mut I, O, O) -> Result<O, E>,
|
||||
),
|
||||
}
|
||||
|
||||
impl<I, O, E> Clone for Infix<I, O, E> {
|
||||
#[inline(always)]
|
||||
fn clone(&self) -> Self {
|
||||
match self {
|
||||
Infix::Left(p, f) => Infix::Left(*p, *f),
|
||||
Infix::Right(p, f) => Infix::Right(*p, *f),
|
||||
Infix::Neither(p, f) => Infix::Neither(*p, *f),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream, O, E: ParserError<I>> Parser<I, Infix<I, O, E>, E> for Infix<I, O, E> {
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<Infix<I, O, E>, E> {
|
||||
empty.value(self.clone()).parse_next(input)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use crate::ascii::digit1;
|
||||
use crate::combinator::fail;
|
||||
use crate::dispatch;
|
||||
use crate::error::ContextError;
|
||||
use crate::token::any;
|
||||
|
||||
use super::*;
|
||||
|
||||
fn parser<'i>() -> impl Parser<&'i str, i32, ContextError> {
|
||||
move |i: &mut &str| {
|
||||
use Infix::*;
|
||||
expression(digit1.parse_to::<i32>())
|
||||
.current_precedence_level(0)
|
||||
.prefix(dispatch! {any;
|
||||
'+' => Prefix(12, |_, a| Ok(a)),
|
||||
'-' => Prefix(12, |_, a: i32| Ok(-a)),
|
||||
_ => fail
|
||||
})
|
||||
.infix(dispatch! {any;
|
||||
'+' => Left(5, |_, a, b| Ok(a + b)),
|
||||
'-' => Left(5, |_, a, b| Ok(a - b)),
|
||||
'*' => Left(7, |_, a, b| Ok(a * b)),
|
||||
'/' => Left(7, |_, a, b| Ok(a / b)),
|
||||
'%' => Left(7, |_, a, b| Ok(a % b)),
|
||||
'^' => Left(9, |_, a, b| Ok(a ^ b)),
|
||||
_ => fail
|
||||
})
|
||||
.parse_next(i)
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_expression() {
|
||||
assert_eq!(parser().parse("-3+-3*4"), Ok(-15));
|
||||
assert_eq!(parser().parse("+2+3*4"), Ok(14));
|
||||
assert_eq!(parser().parse("2*3+4"), Ok(10));
|
||||
}
|
||||
}
|
||||
+800
@@ -0,0 +1,800 @@
|
||||
//! Opaque implementations of [`Parser`]
|
||||
|
||||
use crate::combinator::trace;
|
||||
use crate::combinator::trace_result;
|
||||
use crate::combinator::DisplayDebug;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::error::FromRecoverableError;
|
||||
use crate::error::{AddContext, FromExternalError, ParserError};
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::{Location, Stream};
|
||||
use crate::*;
|
||||
use core::borrow::Borrow;
|
||||
use core::ops::Range;
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::by_ref`]
|
||||
pub struct ByRef<'p, P, I, O, E> {
|
||||
pub(crate) p: &'p mut P,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<I, O, E, P> Parser<I, O, E> for ByRef<'_, P, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E> {
|
||||
self.p.parse_next(i)
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::map`]
|
||||
pub struct Map<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> O2,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) map: G,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, G, I, O, O2, E> Parser<I, O2, E> for Map<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> O2,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O2, E> {
|
||||
match self.parser.parse_next(i) {
|
||||
Err(e) => Err(e),
|
||||
Ok(o) => Ok((self.map)(o)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::try_map`]
|
||||
pub struct TryMap<F, G, I, O, O2, E, E2>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> Result<O2, E2>,
|
||||
I: Stream,
|
||||
E: FromExternalError<I, E2>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) map: G,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
pub(crate) e2: core::marker::PhantomData<E2>,
|
||||
}
|
||||
|
||||
impl<F, G, I, O, O2, E, E2> Parser<I, O2, E> for TryMap<F, G, I, O, O2, E, E2>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> Result<O2, E2>,
|
||||
I: Stream,
|
||||
E: FromExternalError<I, E2>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O2, E> {
|
||||
let start = input.checkpoint();
|
||||
let o = self.parser.parse_next(input)?;
|
||||
let res = (self.map)(o).map_err(|err| {
|
||||
input.reset(&start);
|
||||
E::from_external_error(input, err)
|
||||
});
|
||||
trace_result("verify", &res);
|
||||
res
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::verify_map`]
|
||||
pub struct VerifyMap<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> Option<O2>,
|
||||
I: Stream,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) map: G,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, G, I, O, O2, E> Parser<I, O2, E> for VerifyMap<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> Option<O2>,
|
||||
I: Stream,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O2, E> {
|
||||
let start = input.checkpoint();
|
||||
let o = self.parser.parse_next(input)?;
|
||||
let res = (self.map)(o).ok_or_else(|| {
|
||||
input.reset(&start);
|
||||
ParserError::from_input(input)
|
||||
});
|
||||
trace_result("verify", &res);
|
||||
res
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::and_then`]
|
||||
pub struct AndThen<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: Parser<O, O2, E>,
|
||||
O: StreamIsPartial,
|
||||
I: Stream,
|
||||
{
|
||||
pub(crate) outer: F,
|
||||
pub(crate) inner: G,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, G, I, O, O2, E> Parser<I, O2, E> for AndThen<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: Parser<O, O2, E>,
|
||||
O: StreamIsPartial,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O2, E> {
|
||||
let start = i.checkpoint();
|
||||
let mut o = self.outer.parse_next(i)?;
|
||||
let _ = o.complete();
|
||||
let o2 = self.inner.parse_next(&mut o).map_err(|err| {
|
||||
i.reset(&start);
|
||||
err
|
||||
})?;
|
||||
Ok(o2)
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::parse_to`]
|
||||
pub struct ParseTo<P, I, O, O2, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
O: crate::stream::ParseSlice<O2>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
pub(crate) p: P,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<P, I, O, O2, E> Parser<I, O2, E> for ParseTo<P, I, O, O2, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
O: crate::stream::ParseSlice<O2>,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O2, E> {
|
||||
let start = i.checkpoint();
|
||||
let o = self.p.parse_next(i)?;
|
||||
let res = o.parse_slice().ok_or_else(|| {
|
||||
i.reset(&start);
|
||||
ParserError::from_input(i)
|
||||
});
|
||||
trace_result("verify", &res);
|
||||
res
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::flat_map`]
|
||||
pub struct FlatMap<F, G, H, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> H,
|
||||
H: Parser<I, O2, E>,
|
||||
{
|
||||
pub(crate) f: F,
|
||||
pub(crate) g: G,
|
||||
pub(crate) h: core::marker::PhantomData<H>,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, G, H, I, O, O2, E> Parser<I, O2, E> for FlatMap<F, G, H, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(O) -> H,
|
||||
H: Parser<I, O2, E>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O2, E> {
|
||||
let o = self.f.parse_next(i)?;
|
||||
(self.g)(o).parse_next(i)
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::complete_err`]
|
||||
pub struct CompleteErr<P, I, O, E> {
|
||||
pub(crate) p: P,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<P, I, O, E> Parser<I, O, E> for CompleteErr<P, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O, E> {
|
||||
trace("complete_err", |input: &mut I| {
|
||||
match (self.p).parse_next(input) {
|
||||
Err(err) => match err.needed() {
|
||||
Some(_) => Err(ParserError::from_input(input)),
|
||||
None => Err(err),
|
||||
},
|
||||
rest => rest,
|
||||
}
|
||||
})
|
||||
.parse_next(input)
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::verify`]
|
||||
pub struct Verify<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(&O2) -> bool,
|
||||
I: Stream,
|
||||
O: Borrow<O2>,
|
||||
O2: ?Sized,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) filter: G,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, G, I, O, O2, E> Parser<I, O, E> for Verify<F, G, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(&O2) -> bool,
|
||||
I: Stream,
|
||||
O: Borrow<O2>,
|
||||
O2: ?Sized,
|
||||
E: ParserError<I>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O, E> {
|
||||
let start = input.checkpoint();
|
||||
let o = self.parser.parse_next(input)?;
|
||||
let res = (self.filter)(o.borrow()).then_some(o).ok_or_else(|| {
|
||||
input.reset(&start);
|
||||
ParserError::from_input(input)
|
||||
});
|
||||
trace_result("verify", &res);
|
||||
res
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::value`]
|
||||
pub struct Value<F, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
O2: Clone,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) val: O2,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, O2, E> Parser<I, O2, E> for Value<F, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
O2: Clone,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O2, E> {
|
||||
(self.parser).parse_next(input).map(|_| self.val.clone())
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::default_value`]
|
||||
pub struct DefaultValue<F, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
O2: core::default::Default,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, O2, E> Parser<I, O2, E> for DefaultValue<F, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
O2: core::default::Default,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<O2, E> {
|
||||
(self.parser).parse_next(input).map(|_| O2::default())
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::void`]
|
||||
pub struct Void<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, E> Parser<I, (), E> for Void<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<(), E> {
|
||||
(self.parser).parse_next(input).map(|_| ())
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::take`]
|
||||
pub struct Take<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<I, O, E, F> Parser<I, <I as Stream>::Slice, E> for Take<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<<I as Stream>::Slice, E> {
|
||||
let checkpoint = input.checkpoint();
|
||||
match (self.parser).parse_next(input) {
|
||||
Ok(_) => {
|
||||
let offset = input.offset_from(&checkpoint);
|
||||
input.reset(&checkpoint);
|
||||
let taken = input.next_slice(offset);
|
||||
Ok(taken)
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::with_taken`]
|
||||
pub struct WithTaken<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, E> Parser<I, (O, <I as Stream>::Slice), E> for WithTaken<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<(O, <I as Stream>::Slice), E> {
|
||||
let checkpoint = input.checkpoint();
|
||||
match (self.parser).parse_next(input) {
|
||||
Ok(result) => {
|
||||
let offset = input.offset_from(&checkpoint);
|
||||
input.reset(&checkpoint);
|
||||
let taken = input.next_slice(offset);
|
||||
Ok((result, taken))
|
||||
}
|
||||
Err(e) => Err(e),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::span`]
|
||||
pub struct Span<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream + Location,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<I, O, E, F> Parser<I, Range<usize>, E> for Span<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream + Location,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<Range<usize>, E> {
|
||||
let start = input.current_token_start();
|
||||
self.parser.parse_next(input).map(move |_| {
|
||||
let end = input.previous_token_end();
|
||||
start..end
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::with_span`]
|
||||
pub struct WithSpan<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream + Location,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, E> Parser<I, (O, Range<usize>), E> for WithSpan<F, I, O, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream + Location,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, input: &mut I) -> Result<(O, Range<usize>), E> {
|
||||
let start = input.current_token_start();
|
||||
self.parser.parse_next(input).map(move |output| {
|
||||
let end = input.previous_token_end();
|
||||
(output, (start..end))
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::output_into`]
|
||||
pub struct OutputInto<F, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
O: Into<O2>,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) o2: core::marker::PhantomData<O2>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, O2, E> Parser<I, O2, E> for OutputInto<F, I, O, O2, E>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
O: Into<O2>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O2, E> {
|
||||
self.parser.parse_next(i).map(|o| o.into())
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::err_into`]
|
||||
pub struct ErrInto<F, I, O, E, E2>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
E: Into<E2>,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
pub(crate) e2: core::marker::PhantomData<E2>,
|
||||
}
|
||||
|
||||
impl<F, I, O, E, E2> Parser<I, O, E2> for ErrInto<F, I, O, E, E2>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
E: Into<E2>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E2> {
|
||||
self.parser.parse_next(i).map_err(|err| err.into())
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::context`]
|
||||
pub struct Context<F, I, O, E, C>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: AddContext<I, C>,
|
||||
E: ParserError<I>,
|
||||
C: Clone + core::fmt::Debug,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) context: C,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
impl<F, I, O, E, C> Parser<I, O, E> for Context<F, I, O, E, C>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: AddContext<I, C>,
|
||||
E: ParserError<I>,
|
||||
C: Clone + core::fmt::Debug,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E> {
|
||||
let context = self.context.clone();
|
||||
trace(DisplayDebug(self.context.clone()), move |i: &mut I| {
|
||||
let start = i.checkpoint();
|
||||
(self.parser)
|
||||
.parse_next(i)
|
||||
.map_err(|err| err.add_context(i, &start, context.clone()))
|
||||
})
|
||||
.parse_next(i)
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::context`]
|
||||
pub struct ContextWith<P, I, O, E, F, C, FI>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: AddContext<I, C>,
|
||||
E: ParserError<I>,
|
||||
F: Fn() -> FI + Clone,
|
||||
C: core::fmt::Debug,
|
||||
FI: Iterator<Item = C>,
|
||||
{
|
||||
pub(crate) parser: P,
|
||||
pub(crate) context: F,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
pub(crate) c: core::marker::PhantomData<C>,
|
||||
pub(crate) fi: core::marker::PhantomData<FI>,
|
||||
}
|
||||
|
||||
impl<P, I, O, E, F, C, FI> Parser<I, O, E> for ContextWith<P, I, O, E, F, C, FI>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
I: Stream,
|
||||
E: AddContext<I, C>,
|
||||
E: ParserError<I>,
|
||||
F: Fn() -> FI + Clone,
|
||||
C: core::fmt::Debug,
|
||||
FI: Iterator<Item = C>,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E> {
|
||||
let context = self.context.clone();
|
||||
let start = i.checkpoint();
|
||||
(self.parser).parse_next(i).map_err(|mut err| {
|
||||
for context in context() {
|
||||
err = err.add_context(i, &start, context);
|
||||
}
|
||||
err
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::map_err`]
|
||||
pub struct MapErr<F, G, I, O, E, E2>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(E) -> E2,
|
||||
{
|
||||
pub(crate) parser: F,
|
||||
pub(crate) map: G,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
pub(crate) e2: core::marker::PhantomData<E2>,
|
||||
}
|
||||
|
||||
impl<F, G, I, O, E, E2> Parser<I, O, E2> for MapErr<F, G, I, O, E, E2>
|
||||
where
|
||||
F: Parser<I, O, E>,
|
||||
G: FnMut(E) -> E2,
|
||||
{
|
||||
#[inline]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E2> {
|
||||
match self.parser.parse_next(i) {
|
||||
Err(e) => Err((self.map)(e)),
|
||||
Ok(o) => Ok(o),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::retry_after`]
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
pub struct RetryAfter<P, R, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
R: Parser<I, (), E>,
|
||||
I: Stream,
|
||||
I: Recover<E>,
|
||||
E: ParserError<I> + FromRecoverableError<I, E>,
|
||||
{
|
||||
pub(crate) parser: P,
|
||||
pub(crate) recover: R,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<P, R, I, O, E> Parser<I, O, E> for RetryAfter<P, R, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
R: Parser<I, (), E>,
|
||||
I: Stream,
|
||||
I: Recover<E>,
|
||||
E: ParserError<I> + FromRecoverableError<I, E>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<O, E> {
|
||||
if I::is_recovery_supported() {
|
||||
retry_after_inner(&mut self.parser, &mut self.recover, i)
|
||||
} else {
|
||||
self.parser.parse_next(i)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
fn retry_after_inner<P, R, I, O, E>(parser: &mut P, recover: &mut R, i: &mut I) -> Result<O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
R: Parser<I, (), E>,
|
||||
I: Stream,
|
||||
I: Recover<E>,
|
||||
E: ParserError<I> + FromRecoverableError<I, E>,
|
||||
{
|
||||
loop {
|
||||
let token_start = i.checkpoint();
|
||||
let mut err = match parser.parse_next(i) {
|
||||
Ok(o) => {
|
||||
return Ok(o);
|
||||
}
|
||||
Err(e) if e.is_incomplete() => return Err(e),
|
||||
Err(err) => err,
|
||||
};
|
||||
let err_start = i.checkpoint();
|
||||
let err_start_eof_offset = i.eof_offset();
|
||||
if recover.parse_next(i).is_ok() {
|
||||
let i_eof_offset = i.eof_offset();
|
||||
if err_start_eof_offset == i_eof_offset {
|
||||
// Didn't advance so bubble the error up
|
||||
} else if let Err(err_) = i.record_err(&token_start, &err_start, err) {
|
||||
err = err_;
|
||||
} else {
|
||||
continue;
|
||||
}
|
||||
}
|
||||
|
||||
i.reset(&err_start);
|
||||
err = E::from_recoverable_error(&token_start, &err_start, i, err);
|
||||
return Err(err);
|
||||
}
|
||||
}
|
||||
|
||||
/// [`Parser`] implementation for [`Parser::resume_after`]
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
pub struct ResumeAfter<P, R, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
R: Parser<I, (), E>,
|
||||
I: Stream,
|
||||
I: Recover<E>,
|
||||
E: ParserError<I> + FromRecoverableError<I, E>,
|
||||
{
|
||||
pub(crate) parser: P,
|
||||
pub(crate) recover: R,
|
||||
pub(crate) i: core::marker::PhantomData<I>,
|
||||
pub(crate) o: core::marker::PhantomData<O>,
|
||||
pub(crate) e: core::marker::PhantomData<E>,
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<P, R, I, O, E> Parser<I, Option<O>, E> for ResumeAfter<P, R, I, O, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
R: Parser<I, (), E>,
|
||||
I: Stream,
|
||||
I: Recover<E>,
|
||||
E: ParserError<I> + FromRecoverableError<I, E>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn parse_next(&mut self, i: &mut I) -> Result<Option<O>, E> {
|
||||
if I::is_recovery_supported() {
|
||||
resume_after_inner(&mut self.parser, &mut self.recover, i)
|
||||
} else {
|
||||
self.parser.parse_next(i).map(Some)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
fn resume_after_inner<P, R, I, O, E>(
|
||||
parser: &mut P,
|
||||
recover: &mut R,
|
||||
i: &mut I,
|
||||
) -> Result<Option<O>, E>
|
||||
where
|
||||
P: Parser<I, O, E>,
|
||||
R: Parser<I, (), E>,
|
||||
I: Stream,
|
||||
I: Recover<E>,
|
||||
E: ParserError<I> + FromRecoverableError<I, E>,
|
||||
{
|
||||
let token_start = i.checkpoint();
|
||||
let mut err = match parser.parse_next(i) {
|
||||
Ok(o) => {
|
||||
return Ok(Some(o));
|
||||
}
|
||||
Err(e) if e.is_incomplete() => return Err(e),
|
||||
Err(err) => err,
|
||||
};
|
||||
let err_start = i.checkpoint();
|
||||
if recover.parse_next(i).is_ok() {
|
||||
if let Err(err_) = i.record_err(&token_start, &err_start, err) {
|
||||
err = err_;
|
||||
} else {
|
||||
return Ok(None);
|
||||
}
|
||||
}
|
||||
|
||||
i.reset(&err_start);
|
||||
err = FromRecoverableError::from_recoverable_error(&token_start, &err_start, i, err);
|
||||
Err(err)
|
||||
}
|
||||
+184
@@ -0,0 +1,184 @@
|
||||
//! # List of parsers and combinators
|
||||
//!
|
||||
//! <div class="warning">
|
||||
//!
|
||||
//! **Note**: this list is meant to provide a nicer way to find a parser than reading through the documentation on docs.rs. Function combinators are organized in module so they are a bit easier to find.
|
||||
//!
|
||||
//! </div>
|
||||
//!
|
||||
//! ## Basic elements
|
||||
//!
|
||||
//! Those are used to take a series of tokens for the lowest level elements of your grammar, like, "here is a dot", or "here is an big endian integer".
|
||||
//!
|
||||
//! | combinator | usage | input | new input | output | comment |
|
||||
//! |---|---|---|---|---|---|
|
||||
//! | [`one_of`][crate::token::one_of] | `one_of(['a', 'b', 'c'])` | `"abc"` | `"bc"` | `Ok('a')` |Matches one of the provided [set of tokens][crate::stream::ContainsToken] (works with non ASCII characters too)|
|
||||
//! | [`none_of`][crate::token::none_of] | `none_of(['a', 'b', 'c'])` | `"xyab"` | `"yab"` | `Ok('x')` |Matches anything but one of the provided [set of tokens][crate::stream::ContainsToken]|
|
||||
//! | [`literal`][crate::token::literal] | `"hello"` | `"hello world"` | `" world"` | `Ok("hello")` |Recognizes a specific suite of characters or bytes (see also [`Caseless`][crate::ascii::Caseless])|
|
||||
//! | [`take`][crate::token::take] | `take(4)` | `"hello"` | `"o"` | `Ok("hell")` |Takes a specific number of bytes or characters|
|
||||
//! | [`take_while`][crate::token::take_while] | `take_while(0.., is_alphabetic)` | `"abc123"` | `"123"` | `Ok("abc")` |Returns the longest slice of bytes or characters for which the provided [set of tokens][crate::stream::ContainsToken] matches.|
|
||||
//! | [`take_till`][crate::token::take_till] | `take_till(0.., is_alphabetic)` | `"123abc"` | `"abc"` | `Ok("123")` |Returns a slice of bytes or characters until the provided [set of tokens][crate::stream::ContainsToken] matches. This is the reverse behaviour from `take_while`: `take_till(f)` is equivalent to `take_while(0.., \|c\| !f(c))`|
|
||||
//! | [`take_until`][crate::token::take_until] | `take_until(0.., "world")` | `"Hello world"` | `"world"` | `Ok("Hello ")` |Returns a slice of bytes or characters until the provided [literal][crate::token::literal] is found.|
|
||||
//!
|
||||
//! ## Choice combinators
|
||||
//!
|
||||
//! | combinator | usage | input | new input | output | comment |
|
||||
//! |---|---|---|---|---|---|
|
||||
//! | [`alt`] | `alt(("ab", "cd"))` | `"cdef"` | `"ef"` | `Ok("cd")` |Try a list of parsers and return the result of the first successful one|
|
||||
//! | [`dispatch`] | \- | \- | \- | \- | `match` for parsers |
|
||||
//! | [`permutation`] | `permutation(("ab", "cd", "12"))` | `"cd12abc"` | `"c"` | `Ok(("ab", "cd", "12"))` |Succeeds when all its child parser have succeeded, whatever the order|
|
||||
//!
|
||||
//! ## Sequence combinators
|
||||
//!
|
||||
//! | combinator | usage | input | new input | output | comment |
|
||||
//! |---|---|---|---|---|---|
|
||||
//! | [`(...)` (tuples)][crate::Parser] | `("ab", "XY", take(1))` | `"abXYZ!"` | `"!"` | `Ok(("ab", "XY", "Z"))` |Parse a series of values|
|
||||
//! | [`seq!`] | `seq!(_: '(', take(2), _: ')')` | `"(ab)cd"` | `"cd"` | `Ok("ab")` |Parse a series of values, discarding those you specify|
|
||||
//! | [`delimited`] | `delimited('(', take(2), ')')` | `"(ab)cd"` | `"cd"` | `Ok("ab")` |Parse three values, discarding the first and third value|
|
||||
//! | [`preceded`] | `preceded("ab", "XY")` | `"abXYZ"` | `"Z"` | `Ok("XY")` |Parse two values, discarding the first value|
|
||||
//! | [`terminated`] | `terminated("ab", "XY")` | `"abXYZ"` | `"Z"` | `Ok("ab")` |Parse two values, discarding the second value|
|
||||
//! | [`separated_pair`] | `separated_pair("hello", ',', "world")` | `"hello,world!"` | `"!"` | `Ok(("hello", "world"))` | Parse three values, discarding the middle value|
|
||||
//!
|
||||
//! ## Applying a parser multiple times
|
||||
//!
|
||||
//! | combinator | usage | input | new input | output | comment |
|
||||
//! |---|---|---|---|---|---|
|
||||
//! | [`repeat`] | `repeat(1..=3, "ab")` | `"ababc"` | `"c"` | `Ok(vec!["ab", "ab"])` |Applies the parser between m and n times (n included) and returns the list of results in a Vec|
|
||||
//! | [`repeat_till`] | `repeat_till(0.., "ab", "ef")` | `"ababefg"` | `"g"` | `Ok((vec!["ab", "ab"], "ef"))` |Applies the first parser until the second applies. Returns a tuple containing the list of results from the first in a Vec and the result of the second|
|
||||
//! | [`separated`] | `separated(1..=3, "ab", ",")` | `"ab,ab,ab."` | `"."` | `Ok(vec!["ab", "ab", "ab"])` |Applies the parser and separator between m and n times (n included) and returns the list of results in a Vec|
|
||||
//! | [`Repeat::fold`] | <code>repeat(1..=2, `be_u8`).fold(\|\| 0, \|acc, item\| acc + item)</code> | `[1, 2, 3]` | `[3]` | `Ok(3)` |Applies the parser between m and n times (n included) and folds the list of return value|
|
||||
//!
|
||||
//! ## Partial related
|
||||
//!
|
||||
//! - [`eof`]: Returns its input if it is at the end of input data
|
||||
//! - [`Parser::complete_err`]: Replaces an `Incomplete` returned by the child parser with an `Backtrack`
|
||||
//!
|
||||
//! ## Modifiers
|
||||
//!
|
||||
//! - [`cond`]: Conditional combinator. Wraps another parser and calls it if the condition is met
|
||||
//! - [`Parser::flat_map`]: method to map a new parser from the output of the first parser, then apply that parser over the rest of the input
|
||||
//! - [`Parser::value`]: method to replace the result of a parser
|
||||
//! - [`Parser::default_value`]: method to replace the result of a parser
|
||||
//! - [`Parser::void`]: method to discard the result of a parser
|
||||
//! - [`Parser::map`]: method to map a function on the result of a parser
|
||||
//! - [`Parser::and_then`]: Applies a second parser over the output of the first one
|
||||
//! - [`Parser::verify_map`]: Maps a function returning an `Option` on the output of a parser
|
||||
//! - [`Parser::try_map`]: Maps a function returning a `Result` on the output of a parser
|
||||
//! - [`Parser::parse_to`]: Apply [`std::str::FromStr`] to the output of the parser
|
||||
//! - [`not`]: Returns a result only if the embedded parser returns `Backtrack` or `Incomplete`. Does not consume the input
|
||||
//! - [`opt`]: Make the underlying parser optional
|
||||
//! - [`peek`]: Returns a result without consuming the input
|
||||
//! - [`Parser::take`]: If the child parser was successful, return the consumed input as the produced value
|
||||
//! - [`Parser::with_taken`]: If the child parser was successful, return a tuple of the consumed input and the produced output.
|
||||
//! - [`Parser::span`]: If the child parser was successful, return the location of the consumed input as the produced value
|
||||
//! - [`Parser::with_span`]: If the child parser was successful, return a tuple of the location of the consumed input and the produced output.
|
||||
//! - [`Parser::verify`]: Returns the result of the child parser if it satisfies a verification function
|
||||
//!
|
||||
//! ## Error management and debugging
|
||||
//!
|
||||
//! - [`cut_err`]: Commit the parse result, disallowing alternative parsers from being attempted
|
||||
//! - [`backtrack_err`]: Attempts a parse, allowing alternative parsers to be attempted despite
|
||||
//! use of `cut_err`
|
||||
//! - [`Parser::context`]: Add context to the error if the parser fails
|
||||
//! - [`trace`]: Print the parse state with the `debug` feature flag
|
||||
//! - [`todo()`]: Placeholder parser
|
||||
//!
|
||||
//! ## Remaining combinators
|
||||
//!
|
||||
//! - [`empty`]: Succeed, consuming no input
|
||||
//! - [`fail`]: Inversion of [`empty`]. Always fails.
|
||||
//! - [`Parser::by_ref`]: Allow moving `&mut impl Parser` into other parsers
|
||||
//!
|
||||
//! ## Text parsing
|
||||
//!
|
||||
//! - [`any`][crate::token::any]: Matches one token
|
||||
//! - [`tab`][crate::ascii::tab]: Matches a tab character `\t`
|
||||
//! - [`crlf`][crate::ascii::crlf]: Recognizes the string `\r\n`
|
||||
//! - [`line_ending`][crate::ascii::line_ending]: Recognizes an end of line (both `\n` and `\r\n`)
|
||||
//! - [`newline`][crate::ascii::newline]: Matches a newline character `\n`
|
||||
//! - [`till_line_ending`][crate::ascii::till_line_ending]: Recognizes a string of any char except `\r` or `\n`
|
||||
//! - [`rest`][crate::token::rest]: Return the remaining input
|
||||
//!
|
||||
//! - [`alpha0`][crate::ascii::alpha0]: Recognizes zero or more lowercase and uppercase alphabetic characters: `[a-zA-Z]`. [`alpha1`][crate::ascii::alpha1] does the same but returns at least one character
|
||||
//! - [`alphanumeric0`][crate::ascii::alphanumeric0]: Recognizes zero or more numerical and alphabetic characters: `[0-9a-zA-Z]`. [`alphanumeric1`][crate::ascii::alphanumeric1] does the same but returns at least one character
|
||||
//! - [`space0`][crate::ascii::space0]: Recognizes zero or more spaces and tabs. [`space1`][crate::ascii::space1] does the same but returns at least one character
|
||||
//! - [`multispace0`][crate::ascii::multispace0]: Recognizes zero or more spaces, tabs, carriage returns and line feeds. [`multispace1`][crate::ascii::multispace1] does the same but returns at least one character
|
||||
//! - [`digit0`][crate::ascii::digit0]: Recognizes zero or more numerical characters: `[0-9]`. [`digit1`][crate::ascii::digit1] does the same but returns at least one character
|
||||
//! - [`hex_digit0`][crate::ascii::hex_digit0]: Recognizes zero or more hexadecimal numerical characters: `[0-9A-Fa-f]`. [`hex_digit1`][crate::ascii::hex_digit1] does the same but returns at least one character
|
||||
//! - [`oct_digit0`][crate::ascii::oct_digit0]: Recognizes zero or more octal characters: `[0-7]`. [`oct_digit1`][crate::ascii::oct_digit1] does the same but returns at least one character
|
||||
//!
|
||||
//! - [`float`][crate::ascii::float]: Parse a floating point number in a byte string
|
||||
//! - [`dec_int`][crate::ascii::dec_int]: Decode a variable-width, decimal signed integer
|
||||
//! - [`dec_uint`][crate::ascii::dec_uint]: Decode a variable-width, decimal unsigned integer
|
||||
//! - [`hex_uint`][crate::ascii::hex_uint]: Decode a variable-width, hexadecimal integer
|
||||
//!
|
||||
//! - [`take_escaped`][crate::ascii::take_escaped]: Recognize the input slice with escaped characters
|
||||
//! - [`escaped`][crate::ascii::escaped]: Parse escaped characters, unescaping them
|
||||
//!
|
||||
//! - [`expression()`]: Parse an operator precedence expression with Pratt parsing
|
||||
//!
|
||||
//! ### Character test functions
|
||||
//!
|
||||
//! Use these functions with a combinator like `take_while`:
|
||||
//!
|
||||
//! - [`AsChar::is_alpha`][crate::stream::AsChar::is_alpha]: Tests if byte is ASCII alphabetic: `[A-Za-z]`
|
||||
//! - [`AsChar::is_alphanum`][crate::stream::AsChar::is_alphanum]: Tests if byte is ASCII alphanumeric: `[A-Za-z0-9]`
|
||||
//! - [`AsChar::is_dec_digit`][crate::stream::AsChar::is_dec_digit]: Tests if byte is ASCII digit: `[0-9]`
|
||||
//! - [`AsChar::is_hex_digit`][crate::stream::AsChar::is_hex_digit]: Tests if byte is ASCII hex digit: `[0-9A-Fa-f]`
|
||||
//! - [`AsChar::is_oct_digit`][crate::stream::AsChar::is_oct_digit]: Tests if byte is ASCII octal digit: `[0-7]`
|
||||
//! - [`AsChar::is_space`][crate::stream::AsChar::is_space]: Tests if byte is ASCII space or tab: `[ \t]`
|
||||
//! - [`AsChar::is_newline`][crate::stream::AsChar::is_newline]: Tests if byte is ASCII newline: `[\n]`
|
||||
//!
|
||||
//! ## Binary format parsing
|
||||
//!
|
||||
//! - [`length_repeat`][crate::binary::length_repeat] Gets a number from the first parser, then applies the second parser that many times
|
||||
//! - [`length_take`][crate::binary::length_take]: Gets a number from the first parser, then takes a subslice of the input of that size, and returns that subslice
|
||||
//! - [`length_and_then`][crate::binary::length_and_then]: Gets a number from the first parser, takes a subslice of the input of that size, then applies the second parser on that subslice. If the second parser returns `Incomplete`, `length_value` will return an error
|
||||
//!
|
||||
//! ### Integers
|
||||
//!
|
||||
//! Parsing integers from binary formats can be done in two ways: With parser functions, or combinators with configurable endianness.
|
||||
//!
|
||||
//! - **configurable endianness:** [`i16`][crate::binary::i16], [`i32`][crate::binary::i32],
|
||||
//! [`i64`][crate::binary::i64], [`u16`][crate::binary::u16], [`u32`][crate::binary::u32],
|
||||
//! [`u64`][crate::binary::u64] are combinators that take as argument a
|
||||
//! [`winnow::binary::Endianness`][crate::binary::Endianness], like this: `i16(endianness)`. If the
|
||||
//! parameter is `winnow::binary::Endianness::Big`, parse a big endian `i16` integer, otherwise a
|
||||
//! little endian `i16` integer.
|
||||
//! - **fixed endianness**: The functions are prefixed by `be_` for big endian numbers, and by `le_` for little endian numbers, and the suffix is the type they parse to. As an example, `be_u32` parses a big endian unsigned integer stored in 32 bits.
|
||||
//! - [`be_f32`][crate::binary::be_f32], [`be_f64`][crate::binary::be_f64]: Big endian floating point numbers
|
||||
//! - [`le_f32`][crate::binary::le_f32], [`le_f64`][crate::binary::le_f64]: Little endian floating point numbers
|
||||
//! - [`be_i8`][crate::binary::be_i8], [`be_i16`][crate::binary::be_i16], [`be_i24`][crate::binary::be_i24], [`be_i32`][crate::binary::be_i32], [`be_i64`][crate::binary::be_i64], [`be_i128`][crate::binary::be_i128]: Big endian signed integers
|
||||
//! - [`be_u8`][crate::binary::be_u8], [`be_u16`][crate::binary::be_u16], [`be_u24`][crate::binary::be_u24], [`be_u32`][crate::binary::be_u32], [`be_u64`][crate::binary::be_u64], [`be_u128`][crate::binary::be_u128]: Big endian unsigned integers
|
||||
//! - [`le_i8`][crate::binary::le_i8], [`le_i16`][crate::binary::le_i16], [`le_i24`][crate::binary::le_i24], [`le_i32`][crate::binary::le_i32], [`le_i64`][crate::binary::le_i64], [`le_i128`][crate::binary::le_i128]: Little endian signed integers
|
||||
//! - [`le_u8`][crate::binary::le_u8], [`le_u16`][crate::binary::le_u16], [`le_u24`][crate::binary::le_u24], [`le_u32`][crate::binary::le_u32], [`le_u64`][crate::binary::le_u64], [`le_u128`][crate::binary::le_u128]: Little endian unsigned integers
|
||||
//!
|
||||
//! ### Bit stream parsing
|
||||
//!
|
||||
//! - [`bits`][crate::binary::bits::bits]: Transforms the current input type (byte slice `&[u8]`) to a bit stream on which bit specific parsers and more general combinators can be applied
|
||||
//! - [`bytes`][crate::binary::bits::bytes]: Transforms its bits stream input back into a byte slice for the underlying parser
|
||||
//! - [`take`][crate::binary::bits::take]: Take a set number of bits
|
||||
//! - [`pattern`][crate::binary::bits::pattern]: Check if a set number of bits matches a pattern
|
||||
//! - [`bool`][crate::binary::bits::bool]: Match any one bit
|
||||
|
||||
mod branch;
|
||||
mod core;
|
||||
mod debug;
|
||||
mod expression;
|
||||
mod multi;
|
||||
mod sequence;
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
|
||||
pub mod impls;
|
||||
|
||||
pub use self::branch::*;
|
||||
pub use self::core::*;
|
||||
pub use self::debug::*;
|
||||
pub use self::expression::*;
|
||||
pub use self::multi::*;
|
||||
pub use self::sequence::*;
|
||||
|
||||
#[allow(unused_imports)]
|
||||
use crate::Parser;
|
||||
+1534
File diff suppressed because it is too large.
Load diff
+176
@@ -0,0 +1,176 @@
|
||||
use crate::combinator::trace;
|
||||
use crate::error::ParserError;
|
||||
use crate::stream::Stream;
|
||||
use crate::*;
|
||||
|
||||
#[doc(inline)]
|
||||
pub use crate::seq;
|
||||
#[doc(inline)]
|
||||
pub use crate::unordered_seq;
|
||||
|
||||
/// Sequence two parsers, only returning the output from the second.
|
||||
///
|
||||
/// See also [`seq`] to generalize this across any number of fields.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::error::Needed::Size;
|
||||
/// use winnow::combinator::preceded;
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// preceded("abc", "efg").parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("abcefg"), Ok(("", "efg")));
|
||||
/// assert_eq!(parser.parse_peek("abcefghij"), Ok(("hij", "efg")));
|
||||
/// assert!(parser.parse_peek("").is_err());
|
||||
/// assert!(parser.parse_peek("123").is_err());
|
||||
/// ```
|
||||
#[doc(alias = "ignore_then")]
|
||||
pub fn preceded<Input, Ignored, Output, Error, IgnoredParser, ParseNext>(
|
||||
mut ignored: IgnoredParser,
|
||||
mut parser: ParseNext,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
IgnoredParser: Parser<Input, Ignored, Error>,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
{
|
||||
trace("preceded", move |input: &mut Input| {
|
||||
let _ = ignored.parse_next(input)?;
|
||||
parser.parse_next(input)
|
||||
})
|
||||
}
|
||||
|
||||
/// Sequence two parsers, only returning the output of the first.
|
||||
///
|
||||
/// See also [`seq`] to generalize this across any number of fields.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::error::Needed::Size;
|
||||
/// use winnow::combinator::terminated;
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// terminated("abc", "efg").parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("abcefg"), Ok(("", "abc")));
|
||||
/// assert_eq!(parser.parse_peek("abcefghij"), Ok(("hij", "abc")));
|
||||
/// assert!(parser.parse_peek("").is_err());
|
||||
/// assert!(parser.parse_peek("123").is_err());
|
||||
/// ```
|
||||
#[doc(alias = "then_ignore")]
|
||||
pub fn terminated<Input, Output, Ignored, Error, ParseNext, IgnoredParser>(
|
||||
mut parser: ParseNext,
|
||||
mut ignored: IgnoredParser,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
IgnoredParser: Parser<Input, Ignored, Error>,
|
||||
{
|
||||
trace("terminated", move |input: &mut Input| {
|
||||
let o = parser.parse_next(input)?;
|
||||
ignored.parse_next(input).map(|_| o)
|
||||
})
|
||||
}
|
||||
|
||||
/// Sequence three parsers, only returning the values of the first and third.
|
||||
///
|
||||
/// See also [`seq`] to generalize this across any number of fields.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::error::Needed::Size;
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::separated_pair;
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<(&'i str, &'i str)> {
|
||||
/// separated_pair("abc", "|", "efg").parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("abc|efg"), Ok(("", ("abc", "efg"))));
|
||||
/// assert_eq!(parser.parse_peek("abc|efghij"), Ok(("hij", ("abc", "efg"))));
|
||||
/// assert!(parser.parse_peek("").is_err());
|
||||
/// assert!(parser.parse_peek("123").is_err());
|
||||
/// ```
|
||||
pub fn separated_pair<Input, O1, Sep, O2, Error, P1, SepParser, P2>(
|
||||
mut first: P1,
|
||||
mut sep: SepParser,
|
||||
mut second: P2,
|
||||
) -> impl Parser<Input, (O1, O2), Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
P1: Parser<Input, O1, Error>,
|
||||
SepParser: Parser<Input, Sep, Error>,
|
||||
P2: Parser<Input, O2, Error>,
|
||||
{
|
||||
trace("separated_pair", move |input: &mut Input| {
|
||||
let o1 = first.parse_next(input)?;
|
||||
let _ = sep.parse_next(input)?;
|
||||
second.parse_next(input).map(|o2| (o1, o2))
|
||||
})
|
||||
}
|
||||
|
||||
/// Sequence three parsers, only returning the output of the second.
|
||||
///
|
||||
/// See also [`seq`] to generalize this across any number of fields.
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::error::Needed::Size;
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::delimited;
|
||||
///
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<&'i str> {
|
||||
/// delimited("(", "abc", ")").parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(parser.parse_peek("(abc)"), Ok(("", "abc")));
|
||||
/// assert_eq!(parser.parse_peek("(abc)def"), Ok(("def", "abc")));
|
||||
/// assert!(parser.parse_peek("").is_err());
|
||||
/// assert!(parser.parse_peek("123").is_err());
|
||||
/// ```
|
||||
#[doc(alias = "between")]
|
||||
#[doc(alias = "padded")]
|
||||
pub fn delimited<
|
||||
Input,
|
||||
Ignored1,
|
||||
Output,
|
||||
Ignored2,
|
||||
Error,
|
||||
IgnoredParser1,
|
||||
ParseNext,
|
||||
IgnoredParser2,
|
||||
>(
|
||||
mut ignored1: IgnoredParser1,
|
||||
mut parser: ParseNext,
|
||||
mut ignored2: IgnoredParser2,
|
||||
) -> impl Parser<Input, Output, Error>
|
||||
where
|
||||
Input: Stream,
|
||||
Error: ParserError<Input>,
|
||||
IgnoredParser1: Parser<Input, Ignored1, Error>,
|
||||
ParseNext: Parser<Input, Output, Error>,
|
||||
IgnoredParser2: Parser<Input, Ignored2, Error>,
|
||||
{
|
||||
trace("delimited", move |input: &mut Input| {
|
||||
let _ = ignored1.parse_next(input)?;
|
||||
let o2 = parser.parse_next(input)?;
|
||||
ignored2.parse_next(input).map(|_| o2)
|
||||
})
|
||||
}
|
||||
+4020
File diff suppressed because it is too large.
Load diff
Vendored
+1582
File diff suppressed because it is too large.
Load diff
Vendored
+168
@@ -0,0 +1,168 @@
|
||||
//! > winnow, making parsing a breeze
|
||||
//!
|
||||
//! `winnow` is a parser combinator library
|
||||
//!
|
||||
//! Quick links:
|
||||
//! - [List of combinators][crate::combinator]
|
||||
//! - [Tutorial][_tutorial::chapter_0]
|
||||
//! - [Special Topics][_topic]
|
||||
//! - [Discussions](https://github.com/winnow-rs/winnow/discussions)
|
||||
//! - [CHANGELOG](https://github.com/winnow-rs/winnow/blob/v0.7.15/CHANGELOG.md) (includes major version migration
|
||||
//! guides)
|
||||
//!
|
||||
//! ## Aspirations
|
||||
//!
|
||||
//! `winnow` aims to be your "do everything" parser, much like people treat regular expressions.
|
||||
//!
|
||||
//! In roughly priority order:
|
||||
//! 1. Support writing parser declaratively while not getting in the way of imperative-style
|
||||
//! parsing when needed, working as an open-ended toolbox rather than a close-ended framework.
|
||||
//! 2. Flexible enough to be used for any application, including parsing strings, binary data,
|
||||
//! or separate [lexing and parsing phases][_topic::lexing]
|
||||
//! 3. Zero-cost abstractions, making it easy to write high performance parsers
|
||||
//! 4. Easy to use, making it trivial for one-off uses
|
||||
//!
|
||||
//! In addition:
|
||||
//! - Resilient maintainership, including
|
||||
//! - Willing to break compatibility rather than batching up breaking changes in large releases
|
||||
//! - Leverage feature flags to keep one active branch
|
||||
//! - We will support the last 6 months of rust releases (MSRV, currently 1.64.0)
|
||||
//!
|
||||
//! See also [Special Topic: Why winnow?][crate::_topic::why]
|
||||
//!
|
||||
//! ## Example
|
||||
//!
|
||||
//! Run
|
||||
//! ```console
|
||||
//! $ cargo add winnow
|
||||
//! ```
|
||||
//!
|
||||
//! Then use it to parse:
|
||||
//! ```rust
|
||||
//! # #[cfg(feature = "alloc")] {
|
||||
#![doc = include_str!("../examples/css/parser.rs")]
|
||||
//! # }
|
||||
//! ```
|
||||
//!
|
||||
//! See also the [Tutorial][_tutorial::chapter_0] and [Special Topics][_topic]
|
||||
|
||||
#![cfg_attr(docsrs, feature(doc_cfg))]
|
||||
#![cfg_attr(docsrs, feature(extended_key_value_attributes))]
|
||||
#![cfg_attr(all(not(feature = "std"), not(test)), no_std)]
|
||||
#![warn(missing_docs)]
|
||||
#![warn(clippy::std_instead_of_core)]
|
||||
#![warn(clippy::std_instead_of_alloc)]
|
||||
#![warn(clippy::print_stderr)]
|
||||
#![warn(clippy::print_stdout)]
|
||||
|
||||
#[cfg(feature = "alloc")]
|
||||
#[cfg_attr(test, macro_use)]
|
||||
#[allow(unused_extern_crates)]
|
||||
extern crate alloc;
|
||||
|
||||
#[doc = include_str!("../README.md")]
|
||||
#[cfg(doctest)]
|
||||
pub struct ReadmeDoctests;
|
||||
|
||||
pub(crate) mod util {
|
||||
#[allow(dead_code)]
|
||||
pub(crate) fn from_fn<F: Fn(&mut core::fmt::Formatter<'_>) -> core::fmt::Result>(
|
||||
f: F,
|
||||
) -> FromFn<F> {
|
||||
FromFn(f)
|
||||
}
|
||||
|
||||
pub(crate) struct FromFn<F>(F)
|
||||
where
|
||||
F: Fn(&mut core::fmt::Formatter<'_>) -> core::fmt::Result;
|
||||
|
||||
impl<F> core::fmt::Debug for FromFn<F>
|
||||
where
|
||||
F: Fn(&mut core::fmt::Formatter<'_>) -> core::fmt::Result,
|
||||
{
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
(self.0)(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<F> core::fmt::Display for FromFn<F>
|
||||
where
|
||||
F: Fn(&mut core::fmt::Formatter<'_>) -> core::fmt::Result,
|
||||
{
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
(self.0)(f)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[macro_use]
|
||||
mod macros;
|
||||
|
||||
#[macro_use]
|
||||
pub mod error;
|
||||
|
||||
mod parser;
|
||||
|
||||
pub mod stream;
|
||||
|
||||
pub mod ascii;
|
||||
pub mod binary;
|
||||
pub mod combinator;
|
||||
pub mod token;
|
||||
|
||||
#[cfg(feature = "unstable-doc")]
|
||||
pub mod _topic;
|
||||
#[cfg(feature = "unstable-doc")]
|
||||
pub mod _tutorial;
|
||||
|
||||
/// Core concepts available for glob import
|
||||
///
|
||||
/// Including
|
||||
/// - [`StreamIsPartial`][crate::stream::StreamIsPartial]
|
||||
/// - [`Parser`]
|
||||
///
|
||||
/// ## Example
|
||||
///
|
||||
/// ```rust
|
||||
/// use winnow::prelude::*;
|
||||
///
|
||||
/// fn parse_data(input: &mut &str) -> ModalResult<u64> {
|
||||
/// // ...
|
||||
/// # winnow::ascii::dec_uint(input)
|
||||
/// }
|
||||
///
|
||||
/// fn main() {
|
||||
/// let result = parse_data.parse("100");
|
||||
/// assert_eq!(result, Ok(100));
|
||||
/// }
|
||||
/// ```
|
||||
pub mod prelude {
|
||||
pub use crate::error::ModalError as _;
|
||||
pub use crate::error::ParserError as _;
|
||||
pub use crate::stream::AsChar as _;
|
||||
pub use crate::stream::ContainsToken as _;
|
||||
pub use crate::stream::Stream as _;
|
||||
pub use crate::stream::StreamIsPartial as _;
|
||||
pub use crate::ModalParser;
|
||||
pub use crate::ModalResult;
|
||||
pub use crate::Parser;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
pub use crate::RecoverableParser as _;
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) use crate::TestResult;
|
||||
}
|
||||
|
||||
pub use error::ModalResult;
|
||||
pub use error::Result;
|
||||
pub use parser::*;
|
||||
pub use stream::BStr;
|
||||
pub use stream::Bytes;
|
||||
pub use stream::LocatingSlice;
|
||||
pub use stream::Partial;
|
||||
pub use stream::Stateful;
|
||||
pub use stream::Str;
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) use error::TestResult;
|
||||
+92
@@ -0,0 +1,92 @@
|
||||
/// `match` for parsers
|
||||
///
|
||||
/// While `match` works by accepting a value and returning values:
|
||||
/// ```rust,ignore
|
||||
/// let result_value = match scrutinee_value {
|
||||
/// ArmPattern => arm_value,
|
||||
/// };
|
||||
/// ```
|
||||
/// `dispatch!` composes parsers:
|
||||
/// ```rust,ignore
|
||||
/// let result_parser = dispatch!{scrutinee_parser;
|
||||
/// ArmPattern => arm_parser,
|
||||
/// };
|
||||
/// ```
|
||||
///
|
||||
/// This is useful when parsers have unique prefixes to test for.
|
||||
/// This offers better performance over
|
||||
/// [`alt`][crate::combinator::alt] though it might be at the cost of duplicating parts of your grammar
|
||||
/// if you needed to [`peek(input_parser)`][crate::combinator::peek] the scrutinee.
|
||||
///
|
||||
/// For tight control over the error in a catch-all case, use [`fail`][crate::combinator::fail].
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// use winnow::prelude::*;
|
||||
/// use winnow::combinator::dispatch;
|
||||
/// # use winnow::token::take;
|
||||
/// # use winnow::token::take_while;
|
||||
/// # use winnow::combinator::fail;
|
||||
///
|
||||
/// fn integer(input: &mut &str) -> ModalResult<u64> {
|
||||
/// dispatch! {take(2usize);
|
||||
/// "0b" => take_while(1.., '0'..='1').try_map(|s| u64::from_str_radix(s, 2)),
|
||||
/// "0o" => take_while(1.., '0'..='7').try_map(|s| u64::from_str_radix(s, 8)),
|
||||
/// "0d" => take_while(1.., '0'..='9').try_map(|s| u64::from_str_radix(s, 10)),
|
||||
/// "0x" => take_while(1.., ('0'..='9', 'a'..='f', 'A'..='F')).try_map(|s| u64::from_str_radix(s, 16)),
|
||||
/// _ => fail::<_, u64, _>,
|
||||
/// }
|
||||
/// .parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(integer.parse_peek("0x100 Hello"), Ok((" Hello", 0x100)));
|
||||
/// ```
|
||||
///
|
||||
/// ```rust
|
||||
/// use winnow::prelude::*;
|
||||
/// use winnow::combinator::dispatch;
|
||||
/// # use winnow::token::any;
|
||||
/// # use winnow::combinator::preceded;
|
||||
/// # use winnow::combinator::empty;
|
||||
/// # use winnow::combinator::fail;
|
||||
///
|
||||
/// fn escaped(input: &mut &str) -> ModalResult<char> {
|
||||
/// preceded('\\', escape_seq_char).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// fn escape_seq_char(input: &mut &str) -> ModalResult<char> {
|
||||
/// dispatch! {any;
|
||||
/// 'b' => empty.value('\u{8}'),
|
||||
/// 'f' => empty.value('\u{c}'),
|
||||
/// 'n' => empty.value('\n'),
|
||||
/// 'r' => empty.value('\r'),
|
||||
/// 't' => empty.value('\t'),
|
||||
/// '\\' => empty.value('\\'),
|
||||
/// '"' => empty.value('"'),
|
||||
/// _ => fail::<_, char, _>,
|
||||
/// }
|
||||
/// .parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(escaped.parse_peek("\\nHello"), Ok(("Hello", '\n')));
|
||||
/// ```
|
||||
#[macro_export]
|
||||
#[doc(hidden)] // forced to be visible in intended location
|
||||
macro_rules! dispatch {
|
||||
(
|
||||
$scrutinee_parser:expr;
|
||||
$( $arm_pat:pat $(if $arm_pred:expr)? => $arm_parser: expr ),+ $(,)?
|
||||
) => {
|
||||
$crate::combinator::trace("dispatch", move |i: &mut _|
|
||||
{
|
||||
use $crate::Parser;
|
||||
let initial = $scrutinee_parser.parse_next(i)?;
|
||||
match initial {
|
||||
$(
|
||||
$arm_pat $(if $arm_pred)? => $arm_parser.parse_next(i),
|
||||
)*
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
+58
@@ -0,0 +1,58 @@
|
||||
mod dispatch;
|
||||
mod seq;
|
||||
mod unordered_seq;
|
||||
|
||||
#[cfg(test)]
|
||||
macro_rules! assert_parse(
|
||||
($left: expr, $right: expr) => {
|
||||
let res: $crate::error::ModalResult<_, $crate::error::InputError<_>> = $left;
|
||||
snapbox::assert_data_eq!(snapbox::data::ToDebug::to_debug(&res), $right);
|
||||
};
|
||||
);
|
||||
|
||||
macro_rules! impl_partial_eq {
|
||||
($lhs:ty, $rhs:ty) => {
|
||||
#[allow(unused_lifetimes)]
|
||||
impl<'a> PartialEq<$rhs> for $lhs {
|
||||
#[inline]
|
||||
fn eq(&self, other: &$rhs) -> bool {
|
||||
let l = self;
|
||||
let r: &Self = other.as_ref();
|
||||
PartialEq::eq(l, r)
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(unused_lifetimes)]
|
||||
impl<'a> PartialEq<$lhs> for $rhs {
|
||||
#[inline]
|
||||
fn eq(&self, other: &$lhs) -> bool {
|
||||
PartialEq::eq(other, self)
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
macro_rules! impl_partial_ord {
|
||||
($lhs:ty, $rhs:ty) => {
|
||||
#[allow(unused_lifetimes)]
|
||||
impl<'a> PartialOrd<$rhs> for $lhs {
|
||||
#[inline]
|
||||
fn partial_cmp(&self, other: &$rhs) -> Option<Ordering> {
|
||||
let l = self;
|
||||
let r: &Self = other.as_ref();
|
||||
PartialOrd::partial_cmp(l, r)
|
||||
}
|
||||
}
|
||||
|
||||
#[allow(unused_lifetimes)]
|
||||
impl<'a> PartialOrd<$lhs> for $rhs {
|
||||
#[inline]
|
||||
fn partial_cmp(&self, other: &$lhs) -> Option<Ordering> {
|
||||
PartialOrd::partial_cmp(other, self)
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests;
|
||||
+334
@@ -0,0 +1,334 @@
|
||||
/// Initialize a struct or tuple out of a sequences of parsers
|
||||
///
|
||||
/// Unlike normal struct initialization syntax:
|
||||
/// - `_` fields can exist to run a parser but ignore the result
|
||||
/// - Parse results for a field can later be referenced using the field name
|
||||
///
|
||||
/// Unlike normal tuple initialization syntax:
|
||||
/// - Struct-style initialization (`{ 0: _, 1: _}`) is not supported
|
||||
/// - `_: <parser>` fields can exist to run a parser but ignore the result
|
||||
///
|
||||
///# Example
|
||||
///
|
||||
/// ```
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::ascii::{alphanumeric1, dec_uint, space0};
|
||||
/// # use winnow::combinator::delimited;
|
||||
/// # use winnow::combinator::empty;
|
||||
/// # use winnow::error::ContextError;
|
||||
/// # use winnow::error::ErrMode;
|
||||
/// use winnow::combinator::seq;
|
||||
///
|
||||
/// #[derive(Default, Debug, PartialEq)]
|
||||
/// struct Field {
|
||||
/// namespace: u32,
|
||||
/// name: Vec<u8>,
|
||||
/// value: Vec<u8>,
|
||||
/// point: (u32, u32),
|
||||
/// metadata: Vec<u8>,
|
||||
/// }
|
||||
///
|
||||
/// // Parse into structs / tuple-structs
|
||||
/// fn field(input: &mut &[u8]) -> ModalResult<Field> {
|
||||
/// seq!{Field {
|
||||
/// namespace: empty.value(5),
|
||||
/// name: alphanumeric1.map(|s: &[u8]| s.to_owned()),
|
||||
/// // `_` fields are ignored when building the struct
|
||||
/// _: (space0, b':', space0),
|
||||
/// value: alphanumeric1.map(|s: &[u8]| s.to_owned()),
|
||||
/// _: (space0, b':', space0),
|
||||
/// point: point,
|
||||
/// // default initialization also works
|
||||
/// ..Default::default()
|
||||
/// }}.parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// // Or parse into tuples
|
||||
/// fn point(input: &mut &[u8]) -> ModalResult<(u32, u32)> {
|
||||
/// let mut num = dec_uint::<_, u32, ErrMode<ContextError>>;
|
||||
/// seq!(num, _: (space0, b',', space0), num).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// assert_eq!(
|
||||
/// field.parse_peek(&b"test: data: 123 , 4"[..]),
|
||||
/// Ok((
|
||||
/// &b""[..],
|
||||
/// Field {
|
||||
/// namespace: 5,
|
||||
/// name: b"test"[..].to_owned(),
|
||||
/// value: b"data"[..].to_owned(),
|
||||
/// point: (123, 4),
|
||||
/// metadata: Default::default(),
|
||||
/// },
|
||||
/// )),
|
||||
/// );
|
||||
/// ```
|
||||
#[macro_export]
|
||||
#[doc(alias = "tuple")]
|
||||
#[doc(alias = "preceded")]
|
||||
#[doc(alias = "terminated")]
|
||||
#[doc(alias = "delimited")]
|
||||
#[doc(alias = "pair")]
|
||||
#[doc(alias = "separated_pair")]
|
||||
#[doc(alias = "struct_parser")]
|
||||
#[doc(hidden)] // forced to be visible in intended location
|
||||
macro_rules! seq {
|
||||
($($name: ident)::* { $($fields: tt)* }) => {
|
||||
$crate::combinator::trace(stringify!($($name)::*), move |input: &mut _| {
|
||||
$crate::seq_parse_struct_fields!(
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
input ;
|
||||
);
|
||||
Ok($crate::seq_init_struct_fields!(
|
||||
( $($fields)* );
|
||||
$($name)::* ;
|
||||
))
|
||||
})
|
||||
};
|
||||
($($name: ident)::* ( $($fields: tt)* )) => {
|
||||
$crate::combinator::trace(stringify!($($name)::*), move |input: &mut _| {
|
||||
$crate::seq_parse_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
input;
|
||||
);
|
||||
Ok($crate::seq_init_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
$($name)::*;
|
||||
))
|
||||
})
|
||||
};
|
||||
(( $($fields: tt)* )) => {
|
||||
$crate::combinator::trace("tuple", move |input: &mut _| {
|
||||
$crate::seq_parse_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
input;
|
||||
);
|
||||
Ok($crate::seq_init_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
;
|
||||
))
|
||||
})
|
||||
};
|
||||
($($fields: tt)*) => {
|
||||
$crate::seq!((
|
||||
$($fields)*
|
||||
))
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! seq_parse_struct_fields {
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$input: ident ;
|
||||
) => {
|
||||
let $unnamed1 = $crate::Parser::parse_next(&mut $head_parser, $input)?;
|
||||
$crate::seq_parse_struct_fields!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
$input ;
|
||||
)
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$input: ident ;
|
||||
) => {
|
||||
let $unnamed1 = $crate::Parser::parse_next(&mut $head_parser, $input)?;
|
||||
};
|
||||
(
|
||||
( $head_field: ident : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$input: ident ;
|
||||
) => {
|
||||
let $head_field = $crate::Parser::parse_next(&mut $head_parser, $input)?;
|
||||
$crate::seq_parse_struct_fields!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
$input ;
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_field: ident : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$input: ident ;
|
||||
) => {
|
||||
let $head_field = $crate::Parser::parse_next(&mut $head_parser, $input)?;
|
||||
};
|
||||
(
|
||||
( .. $update: expr );
|
||||
( $($unnamed: ident),* );
|
||||
$input: expr ;
|
||||
) => {};
|
||||
(
|
||||
( $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
$input: expr ;
|
||||
) => {};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! seq_parse_tuple_fields {
|
||||
(
|
||||
( $(_ :)? $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$input: ident;
|
||||
) => {
|
||||
let $unnamed1 = $crate::Parser::parse_next(&mut $head_parser, $input)?;
|
||||
$crate::seq_parse_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
$input ;
|
||||
)
|
||||
};
|
||||
(
|
||||
( $(_ :)? $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$input: ident;
|
||||
) => {
|
||||
let $unnamed1 = $crate::Parser::parse_next(&mut $head_parser, $input)?;
|
||||
};
|
||||
(
|
||||
( $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
$input: expr;
|
||||
) => {};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! seq_init_struct_fields {
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
$($name: ident)::* ;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::seq_init_struct_fields!(
|
||||
( $($fields)* );
|
||||
$($name)::* ;
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr );
|
||||
$($name: ident)::* ;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::seq_init_struct_fields!(
|
||||
();
|
||||
$($name)::* ;
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_field: ident : $head_parser: expr, $($fields: tt)* );
|
||||
$($name: ident)::* ;
|
||||
$($inits: tt)*
|
||||
) =>
|
||||
{
|
||||
$crate::seq_init_struct_fields!(
|
||||
( $($fields)* );
|
||||
$($name)::* ;
|
||||
$($inits)* $head_field,
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_field: ident : $head_parser: expr );
|
||||
$($name: ident)::* ;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::seq_init_struct_fields!(
|
||||
();
|
||||
$($name)::* ;
|
||||
$($inits)* $head_field,
|
||||
)
|
||||
};
|
||||
(
|
||||
( .. $update: expr );
|
||||
$($name: ident)::* ;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$($name)::* { $($inits)* ..$update }
|
||||
};
|
||||
(
|
||||
( $(,)? );
|
||||
$($name: ident)::* ;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$($name)::* { $($inits)* }
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! seq_init_tuple_fields {
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$($name: ident)::*;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::seq_init_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
$($name)::* ;
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$($name: ident)::*;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::seq_init_tuple_fields!(
|
||||
();
|
||||
( $($unnamed),* );
|
||||
$($name)::* ;
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$($name: ident)::*;
|
||||
$($inits: tt)*
|
||||
) =>
|
||||
{
|
||||
$crate::seq_init_tuple_fields!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
$($name)::* ;
|
||||
$($inits)* $unnamed1,
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$($name: ident)::*;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::seq_init_tuple_fields!(
|
||||
();
|
||||
( $($unnamed),* );
|
||||
$($name)::* ;
|
||||
$($inits)* $unnamed1,
|
||||
)
|
||||
};
|
||||
(
|
||||
( $(,)? );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
$($name: ident)::*;
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$($name)::* ( $($inits)* )
|
||||
};
|
||||
}
|
||||
+1330
File diff suppressed because it is too large.
Load diff
+592
@@ -0,0 +1,592 @@
|
||||
/// Initialize a struct or tuple out of an unordered sequences of parsers
|
||||
///
|
||||
/// Unlike normal struct initialization syntax:
|
||||
/// - `_` fields can exist to run a parser but ignore the result
|
||||
/// - Parse results for a field can later be referenced using the field name
|
||||
///
|
||||
/// Unlike normal tuple initialization syntax:
|
||||
/// - Struct-style initialization (`{ 0: _, 1: _}`) is not supported
|
||||
/// - `_: <parser>` fields can exist to run a parser but ignore the result
|
||||
///
|
||||
/// To stop on an error, rather than trying further permutations, see
|
||||
/// [`cut_err`][crate::combinator::cut_err] ([example][crate::_tutorial::chapter_7]).
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{error::ErrMode, error::Needed};
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::ascii::{alpha1, digit1};
|
||||
/// use winnow::combinator::unordered_seq;
|
||||
/// # fn main() {
|
||||
/// fn parser<'i>(input: &mut &'i str) -> ModalResult<(&'i str, &'i str)> {
|
||||
/// unordered_seq!((alpha1, digit1)).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// // permutation takes alphabetic characters then digit
|
||||
/// assert_eq!(parser.parse_peek("abc123"), Ok(("", ("abc", "123"))));
|
||||
///
|
||||
/// // but also in inverse order
|
||||
/// assert_eq!(parser.parse_peek("123abc"), Ok(("", ("abc", "123"))));
|
||||
///
|
||||
/// // it will fail if one of the parsers failed
|
||||
/// assert!(parser.parse_peek("abc;").is_err());
|
||||
/// # }
|
||||
/// ```
|
||||
///
|
||||
/// The parsers are applied greedily: if there are multiple unapplied parsers
|
||||
/// that could parse the next slice of input, the first one is used.
|
||||
/// ```rust
|
||||
/// # use winnow::error::ErrMode;
|
||||
/// # use winnow::prelude::*;
|
||||
/// use winnow::combinator::unordered_seq;
|
||||
/// use winnow::token::any;
|
||||
///
|
||||
/// fn parser(input: &mut &str) -> ModalResult<(char, char)> {
|
||||
/// unordered_seq!((any, 'a')).parse_next(input)
|
||||
/// }
|
||||
///
|
||||
/// // any parses 'b', then char('a') parses 'a'
|
||||
/// assert_eq!(parser.parse_peek("ba"), Ok(("", ('b', 'a'))));
|
||||
///
|
||||
/// // any parses 'a', then char('a') fails on 'b',
|
||||
/// // even though char('a') followed by any would succeed
|
||||
/// assert!(parser.parse_peek("ab").is_err());
|
||||
/// ```
|
||||
#[macro_export]
|
||||
#[doc(alias = "permutation")]
|
||||
#[doc(hidden)] // forced to be visible in intended location
|
||||
macro_rules! unordered_seq {
|
||||
($($name: ident)::* { $($fields: tt)* }) => {
|
||||
$crate::combinator::trace(stringify!($($name)::*), move |input: &mut _| {
|
||||
$crate::unordered_seq_parse_struct_body!(
|
||||
( $($name)::* );
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
( input );
|
||||
)
|
||||
})
|
||||
};
|
||||
(( $($fields: tt)* )) => {
|
||||
$crate::combinator::trace("tuple", move |input: &mut _| {
|
||||
$crate::unordered_seq_parse_tuple_body!(
|
||||
( );
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
( input );
|
||||
)
|
||||
})
|
||||
};
|
||||
($($name: ident)::* ( $($fields: tt)* )) => {
|
||||
$crate::combinator::trace(stringify!($($name)::*), move |input: &mut _| {
|
||||
$crate::unordered_seq_parse_tuple_body!(
|
||||
( $($name)::* );
|
||||
( $($fields)* );
|
||||
( _0, _1, _2, _3, _4, _5, _6, _7, _8, _9, _10, _11, _12, _13, _14, _15, _16, _17, _18, _19, _20 );
|
||||
( input );
|
||||
)
|
||||
})
|
||||
};
|
||||
($($fields: tt)*) => {
|
||||
$crate::unordered_seq!((
|
||||
$($fields)*
|
||||
))
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_struct_body {
|
||||
(
|
||||
( $($name: ident)::* );
|
||||
( $($fields: tt)* );
|
||||
( $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
) => {{
|
||||
// Workaround a type inference issue by hinting to the compiler `I` within `ParserError` calls
|
||||
fn recover_err<I, E>(_input: &I, curr: E, acc: &mut Option<E>) -> Option<E>
|
||||
where
|
||||
I: $crate::stream::Stream,
|
||||
E: $crate::error::ParserError<I>,
|
||||
{
|
||||
if $crate::error::ParserError::is_backtrack(&curr) {
|
||||
*acc = Some(match acc.take() {
|
||||
Some(err) => $crate::error::ParserError::or(err, curr),
|
||||
None => curr,
|
||||
});
|
||||
None
|
||||
} else {
|
||||
Some(curr)
|
||||
}
|
||||
}
|
||||
|
||||
$crate::unordered_seq_parse_struct_init!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
);
|
||||
|
||||
loop {
|
||||
let mut err = ::core::option::Option::None;
|
||||
let start = $crate::stream::Stream::checkpoint($input);
|
||||
|
||||
$crate::unordered_seq_parse_struct_each!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $input );
|
||||
( start );
|
||||
( err );
|
||||
);
|
||||
|
||||
// If we reach here, every iterator has either been applied before,
|
||||
// or errored on the remaining input
|
||||
if let ::core::option::Option::Some(err) = err {
|
||||
// There are remaining parsers, and all errored on the remaining input
|
||||
break Err($crate::error::ParserError::append(err, $input, &start));
|
||||
}
|
||||
|
||||
// All parsers were applied
|
||||
break Ok(
|
||||
$crate::unordered_seq_parse_struct_collect!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
)
|
||||
);
|
||||
}
|
||||
}}
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_struct_init {
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
) => {
|
||||
let mut $unnamed1 = ::core::option::Option::None;
|
||||
$crate::unordered_seq_parse_struct_init!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
);
|
||||
};
|
||||
(
|
||||
( $named: ident : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
) => {
|
||||
let mut $named = ::core::option::Option::None;
|
||||
$crate::unordered_seq_parse_struct_init!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
);
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
) => {
|
||||
let mut $unnamed1 = ::core::option::Option::None;
|
||||
};
|
||||
(
|
||||
( $named: ident : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
) => {
|
||||
let mut $named = ::core::option::Option::None;
|
||||
};
|
||||
(
|
||||
( .. $update: expr $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
) => {
|
||||
};
|
||||
(
|
||||
( $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
) => {
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_struct_collect {
|
||||
(
|
||||
( $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$($name)::* { $($inits)* }
|
||||
};
|
||||
(
|
||||
( .. $update: expr $(,)? );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_struct_collect!(
|
||||
( , );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)* .. $update
|
||||
)
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_struct_collect!(
|
||||
( , );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_struct_collect!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( $named: ident : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_struct_collect!(
|
||||
( , );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)* $named: $named.unwrap(),
|
||||
)
|
||||
};
|
||||
(
|
||||
( $named: ident : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_struct_collect!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)* $named: $named.unwrap(),
|
||||
)
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_struct_each(
|
||||
(
|
||||
( _ : $head_parser: expr $(,)? );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
$crate::unordered_seq_parse_next!(
|
||||
( $head_parser );
|
||||
( $unnamed1 );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
);
|
||||
(
|
||||
( $named: ident : $head_parser: expr $(,)? );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
$crate::unordered_seq_parse_next!(
|
||||
( $head_parser );
|
||||
( $named );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
);
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
$crate::unordered_seq_parse_next!(
|
||||
( $head_parser );
|
||||
( $unnamed1 );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
$crate::unordered_seq_parse_struct_each!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
);
|
||||
(
|
||||
( $named: ident : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
$crate::unordered_seq_parse_next!(
|
||||
( $head_parser );
|
||||
( $named );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
$crate::unordered_seq_parse_struct_each!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
);
|
||||
(
|
||||
( .. $update: expr $(,)? );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
);
|
||||
);
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_tuple_body {
|
||||
(
|
||||
( $($name: ident)::* );
|
||||
( $($fields: tt)* );
|
||||
( $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
) => {{
|
||||
// Workaround a type inference issue by hinting to the compiler `I` within `ParserError` calls
|
||||
fn recover_err<I, E>(_input: &I, curr: E, acc: &mut Option<E>) -> Option<E>
|
||||
where
|
||||
I: $crate::stream::Stream,
|
||||
E: $crate::error::ParserError<I>,
|
||||
{
|
||||
if $crate::error::ParserError::is_backtrack(&curr) {
|
||||
*acc = Some(match acc.take() {
|
||||
Some(err) => $crate::error::ParserError::or(err, curr),
|
||||
None => curr,
|
||||
});
|
||||
None
|
||||
} else {
|
||||
Some(curr)
|
||||
}
|
||||
}
|
||||
|
||||
$crate::unordered_seq_parse_tuple_init!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
);
|
||||
|
||||
loop {
|
||||
let mut err = ::core::option::Option::None;
|
||||
let start = $crate::stream::Stream::checkpoint($input);
|
||||
|
||||
$crate::unordered_seq_parse_tuple_each!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $input );
|
||||
( start );
|
||||
( err );
|
||||
);
|
||||
|
||||
// If we reach here, every iterator has either been applied before,
|
||||
// or errored on the remaining input
|
||||
if let ::core::option::Option::Some(err) = err {
|
||||
// There are remaining parsers, and all errored on the remaining input
|
||||
break Err($crate::error::ParserError::append(err, $input, &start));
|
||||
}
|
||||
|
||||
// All parsers were applied
|
||||
break Ok(
|
||||
$crate::unordered_seq_parse_tuple_collect!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
)
|
||||
);
|
||||
}
|
||||
}}
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_tuple_init {
|
||||
(
|
||||
( $(_ :)? $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
) => {
|
||||
let mut $unnamed1 = ::core::option::Option::None;
|
||||
$crate::unordered_seq_parse_tuple_init!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
);
|
||||
};
|
||||
(
|
||||
( $(_ :)? $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
) => {
|
||||
let mut $unnamed1 = ::core::option::Option::None;
|
||||
};
|
||||
(
|
||||
( $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
) => {
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_tuple_collect {
|
||||
(
|
||||
( $(,)? );
|
||||
( $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$($name)::* ( $($inits)* )
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_tuple_collect!(
|
||||
( , );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( _ : $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_tuple_collect!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)*
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_parser: expr );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_tuple_collect!(
|
||||
( , );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)* $unnamed1.unwrap(),
|
||||
)
|
||||
};
|
||||
(
|
||||
( $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $($name: ident)::* );
|
||||
$($inits: tt)*
|
||||
) => {
|
||||
$crate::unordered_seq_parse_tuple_collect!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $($name)::* );
|
||||
$($inits)* $unnamed1.unwrap(),
|
||||
)
|
||||
};
|
||||
}
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_tuple_each(
|
||||
(
|
||||
( $(_ :)? $head_parser: expr $(,)? );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
$crate::unordered_seq_parse_next!(
|
||||
( $head_parser );
|
||||
( $unnamed1 );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
);
|
||||
(
|
||||
( $(_ :)? $head_parser: expr, $($fields: tt)* );
|
||||
( $unnamed1: ident, $($unnamed: ident),* );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
$crate::unordered_seq_parse_next!(
|
||||
( $head_parser );
|
||||
( $unnamed1 );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
$crate::unordered_seq_parse_tuple_each!(
|
||||
( $($fields)* );
|
||||
( $($unnamed),* );
|
||||
( $input );
|
||||
( $start );
|
||||
( $err );
|
||||
);
|
||||
);
|
||||
);
|
||||
|
||||
#[macro_export]
|
||||
#[doc(hidden)]
|
||||
macro_rules! unordered_seq_parse_next(
|
||||
(
|
||||
( $field: tt );
|
||||
( $unnamed: ident );
|
||||
( $input: expr );
|
||||
( $start: expr );
|
||||
( $err: expr );
|
||||
) => (
|
||||
if $unnamed.is_none() {
|
||||
$crate::stream::Stream::reset($input, &$start);
|
||||
|
||||
match $crate::Parser::parse_next(&mut $field, $input) {
|
||||
Ok(o) => {
|
||||
$unnamed = Some(o);
|
||||
continue;
|
||||
}
|
||||
Err(e) => {
|
||||
if let Some(cut) = recover_err($input, e, &mut $err) {
|
||||
return Err(cut);
|
||||
}
|
||||
}
|
||||
};
|
||||
}
|
||||
);
|
||||
);
|
||||
Vendored
+1592
File diff suppressed because it is too large.
Load diff
+541
@@ -0,0 +1,541 @@
|
||||
use core::num::NonZeroUsize;
|
||||
|
||||
use crate::error::Needed;
|
||||
use crate::stream::AsBStr;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::FindSlice;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
use core::iter::{Cloned, Enumerate};
|
||||
use core::slice::Iter;
|
||||
use core::{cmp::Ordering, fmt, ops};
|
||||
|
||||
/// Improved `Debug` experience for `&[u8]` UTF-8-ish streams
|
||||
#[allow(clippy::derived_hash_with_manual_eq)]
|
||||
#[derive(Hash)]
|
||||
#[repr(transparent)]
|
||||
pub struct BStr([u8]);
|
||||
|
||||
impl BStr {
|
||||
/// Make a stream out of a byte slice-like.
|
||||
#[inline]
|
||||
pub fn new<B: ?Sized + AsRef<[u8]>>(bytes: &B) -> &Self {
|
||||
Self::from_bytes(bytes.as_ref())
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn from_bytes(slice: &[u8]) -> &Self {
|
||||
unsafe { core::mem::transmute(slice) }
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl SliceLen for &BStr {
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'i> Stream for &'i BStr {
|
||||
type Token = u8;
|
||||
type Slice = &'i [u8];
|
||||
|
||||
type IterOffsets = Enumerate<Cloned<Iter<'i, u8>>>;
|
||||
|
||||
type Checkpoint = Checkpoint<Self, Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.iter().cloned().enumerate()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.len()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
if self.is_empty() {
|
||||
None
|
||||
} else {
|
||||
let token = self[0];
|
||||
*self = &self[1..];
|
||||
Some(token)
|
||||
}
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
if self.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(self[0])
|
||||
}
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.iter().position(|b| predicate(*b))
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
if let Some(needed) = tokens.checked_sub(self.len()).and_then(NonZeroUsize::new) {
|
||||
Err(Needed::Size(needed))
|
||||
} else {
|
||||
Ok(tokens)
|
||||
}
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
let (slice, next) = self.0.split_at(offset);
|
||||
*self = BStr::from_bytes(next);
|
||||
slice
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
#[cfg(debug_assertions)]
|
||||
self.peek_slice(offset);
|
||||
|
||||
// SAFETY: `Stream::next_slice_unchecked` requires `offset` to be in bounds
|
||||
let slice = unsafe { self.0.get_unchecked(..offset) };
|
||||
// SAFETY: `Stream::next_slice_unchecked` requires `offset` to be in bounds
|
||||
let next = unsafe { self.0.get_unchecked(offset..) };
|
||||
*self = BStr::from_bytes(next);
|
||||
slice
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
&self[..offset]
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
#[cfg(debug_assertions)]
|
||||
self.peek_slice(offset);
|
||||
|
||||
// SAFETY: `Stream::next_slice_unchecked` requires `offset` to be in bounds
|
||||
let slice = unsafe { self.0.get_unchecked(..offset) };
|
||||
slice
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(*self)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
*self = checkpoint.inner;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<E> Recover<E> for &BStr {
|
||||
#[inline(always)]
|
||||
fn record_err(
|
||||
&mut self,
|
||||
_token_start: &Self::Checkpoint,
|
||||
_err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
Err(err)
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl StreamIsPartial for &BStr {
|
||||
type PartialState = ();
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
// Already complete
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, _state: Self::PartialState) {}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl Offset for &BStr {
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, start: &Self) -> usize {
|
||||
self.as_bytes().offset_from(&start.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Offset<<&'a BStr as Stream>::Checkpoint> for &'a BStr {
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<&'a BStr as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl AsBStr for &BStr {
|
||||
#[inline(always)]
|
||||
fn as_bstr(&self) -> &[u8] {
|
||||
(*self).as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a, T> Compare<T> for &'a BStr
|
||||
where
|
||||
&'a [u8]: Compare<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn compare(&self, t: T) -> CompareResult {
|
||||
let bytes = (*self).as_bytes();
|
||||
bytes.compare(t)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'i, S> FindSlice<S> for &'i BStr
|
||||
where
|
||||
&'i [u8]: FindSlice<S>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn find_slice(&self, substr: S) -> Option<core::ops::Range<usize>> {
|
||||
let bytes = (*self).as_bytes();
|
||||
let offset = bytes.find_slice(substr);
|
||||
offset
|
||||
}
|
||||
}
|
||||
|
||||
impl UpdateSlice for &BStr {
|
||||
#[inline(always)]
|
||||
fn update_slice(self, inner: Self::Slice) -> Self {
|
||||
BStr::new(inner)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "alloc")]
|
||||
impl fmt::Display for BStr {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
alloc::string::String::from_utf8_lossy(self.as_bytes()).fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Debug for BStr {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
if !f.alternate() {
|
||||
write!(f, "\"")?;
|
||||
}
|
||||
for byte in self.as_bytes() {
|
||||
let c = *byte as char;
|
||||
write!(f, "{}", c.escape_debug())?;
|
||||
}
|
||||
if !f.alternate() {
|
||||
write!(f, "\"")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Deref for BStr {
|
||||
type Target = [u8];
|
||||
|
||||
#[inline]
|
||||
fn deref(&self) -> &[u8] {
|
||||
self.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<usize> for BStr {
|
||||
type Output = u8;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, idx: usize) -> &u8 {
|
||||
&self.as_bytes()[idx]
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeFull> for BStr {
|
||||
type Output = BStr;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, _: ops::RangeFull) -> &BStr {
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::Range<usize>> for BStr {
|
||||
type Output = BStr;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::Range<usize>) -> &BStr {
|
||||
BStr::new(&self.as_bytes()[r.start..r.end])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeInclusive<usize>> for BStr {
|
||||
type Output = BStr;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeInclusive<usize>) -> &BStr {
|
||||
BStr::new(&self.as_bytes()[*r.start()..=*r.end()])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeFrom<usize>> for BStr {
|
||||
type Output = BStr;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeFrom<usize>) -> &BStr {
|
||||
BStr::new(&self.as_bytes()[r.start..])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeTo<usize>> for BStr {
|
||||
type Output = BStr;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeTo<usize>) -> &BStr {
|
||||
BStr::new(&self.as_bytes()[..r.end])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeToInclusive<usize>> for BStr {
|
||||
type Output = BStr;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeToInclusive<usize>) -> &BStr {
|
||||
BStr::new(&self.as_bytes()[..=r.end])
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<[u8]> for BStr {
|
||||
#[inline]
|
||||
fn as_ref(&self) -> &[u8] {
|
||||
self.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<BStr> for [u8] {
|
||||
#[inline]
|
||||
fn as_ref(&self) -> &BStr {
|
||||
BStr::new(self)
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<BStr> for str {
|
||||
#[inline]
|
||||
fn as_ref(&self) -> &BStr {
|
||||
BStr::new(self)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "alloc")]
|
||||
impl alloc::borrow::ToOwned for BStr {
|
||||
type Owned = alloc::vec::Vec<u8>;
|
||||
|
||||
#[inline]
|
||||
fn to_owned(&self) -> Self::Owned {
|
||||
alloc::vec::Vec::from(self.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "alloc")]
|
||||
impl core::borrow::Borrow<BStr> for alloc::vec::Vec<u8> {
|
||||
#[inline]
|
||||
fn borrow(&self) -> &BStr {
|
||||
BStr::from_bytes(self.as_slice())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Default for &'a BStr {
|
||||
fn default() -> &'a BStr {
|
||||
BStr::new(b"")
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> From<&'a [u8]> for &'a BStr {
|
||||
#[inline]
|
||||
fn from(s: &'a [u8]) -> &'a BStr {
|
||||
BStr::new(s)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> From<&'a BStr> for &'a [u8] {
|
||||
#[inline]
|
||||
fn from(s: &'a BStr) -> &'a [u8] {
|
||||
BStr::as_bytes(s)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> From<&'a str> for &'a BStr {
|
||||
#[inline]
|
||||
fn from(s: &'a str) -> &'a BStr {
|
||||
BStr::new(s.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
impl Eq for BStr {}
|
||||
|
||||
impl PartialEq<BStr> for BStr {
|
||||
#[inline]
|
||||
fn eq(&self, other: &BStr) -> bool {
|
||||
self.as_bytes() == other.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl_partial_eq!(BStr, [u8]);
|
||||
impl_partial_eq!(BStr, &'a [u8]);
|
||||
impl_partial_eq!(BStr, str);
|
||||
impl_partial_eq!(BStr, &'a str);
|
||||
|
||||
impl PartialOrd for BStr {
|
||||
#[inline]
|
||||
fn partial_cmp(&self, other: &BStr) -> Option<Ordering> {
|
||||
Some(self.cmp(other))
|
||||
}
|
||||
}
|
||||
|
||||
impl Ord for BStr {
|
||||
#[inline]
|
||||
fn cmp(&self, other: &BStr) -> Ordering {
|
||||
Ord::cmp(self.as_bytes(), other.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
impl_partial_ord!(BStr, [u8]);
|
||||
impl_partial_ord!(BStr, &'a [u8]);
|
||||
impl_partial_ord!(BStr, str);
|
||||
impl_partial_ord!(BStr, &'a str);
|
||||
|
||||
#[cfg(test)]
|
||||
mod test {
|
||||
use crate::stream::BStr;
|
||||
|
||||
#[test]
|
||||
fn partial_eq_bstr_byte_slice() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = BStr::new(input);
|
||||
assert!(actual == input);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_eq_byte_slice_bstr() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = BStr::new(input);
|
||||
assert!(input == actual);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_eq_bstr_str() {
|
||||
let input = "foo";
|
||||
let actual = BStr::new(input);
|
||||
assert!(actual == input);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_eq_str_bstr() {
|
||||
let input = "foo";
|
||||
let actual = BStr::new(input);
|
||||
assert!(input == actual);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_bstr_byte_slice() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = BStr::new(input);
|
||||
assert!(actual.partial_cmp(input) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_byte_slice_bstr() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = BStr::new(input);
|
||||
assert!(input.partial_cmp(actual) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_bstr_str() {
|
||||
let input = "foo";
|
||||
let actual = BStr::new(input);
|
||||
assert!(actual.partial_cmp(input) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_str_bstr() {
|
||||
let input = "foo";
|
||||
let actual = BStr::new(input);
|
||||
assert!(input.partial_cmp(actual) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "std"))]
|
||||
mod display {
|
||||
use crate::stream::BStr;
|
||||
|
||||
#[test]
|
||||
fn clean() {
|
||||
assert_eq!(&format!("{}", BStr::new(b"abc")), "abc");
|
||||
assert_eq!(&format!("{}", BStr::new(b"\xf0\x28\x8c\xbc")), "�(��");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "std"))]
|
||||
mod debug {
|
||||
use crate::stream::BStr;
|
||||
use crate::stream::Stream as _;
|
||||
use snapbox::assert_data_eq;
|
||||
use snapbox::str;
|
||||
|
||||
#[test]
|
||||
fn test_debug() {
|
||||
let input = BStr::new(b"abc");
|
||||
let expected = str![[r#""abc""#]];
|
||||
assert_data_eq!(&format!("{input:?}"), expected);
|
||||
|
||||
let input = BStr::new(b"\0\0\0 ftypisom\0\0\x02\0isomiso2avc1mp");
|
||||
let expected = str![[r#""/0/0/0 ftypisom/0/0/u{2}/0isomiso2avc1mp""#]];
|
||||
assert_data_eq!(&format!("{input:?}"), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pretty_debug() {
|
||||
let input = BStr::new(b"abc");
|
||||
let expected = str!["abc"];
|
||||
assert_data_eq!(&format!("{input:#?}"), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_trace() {
|
||||
let input = BStr::new(b"abc");
|
||||
let expected = str!["abc"];
|
||||
assert_data_eq!(
|
||||
crate::util::from_fn(|f| input.trace(f)).to_string(),
|
||||
expected
|
||||
);
|
||||
}
|
||||
}
|
||||
+563
@@ -0,0 +1,563 @@
|
||||
use core::num::NonZeroUsize;
|
||||
|
||||
use crate::error::Needed;
|
||||
use crate::stream::AsBytes;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::FindSlice;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
use core::iter::{Cloned, Enumerate};
|
||||
use core::slice::Iter;
|
||||
use core::{cmp::Ordering, fmt, ops};
|
||||
|
||||
/// Improved `Debug` experience for `&[u8]` byte streams
|
||||
#[allow(clippy::derived_hash_with_manual_eq)]
|
||||
#[derive(Hash)]
|
||||
#[repr(transparent)]
|
||||
pub struct Bytes([u8]);
|
||||
|
||||
impl Bytes {
|
||||
/// Make a stream out of a byte slice-like.
|
||||
#[inline]
|
||||
pub fn new<B: ?Sized + AsRef<[u8]>>(bytes: &B) -> &Self {
|
||||
Self::from_bytes(bytes.as_ref())
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn from_bytes(slice: &[u8]) -> &Self {
|
||||
unsafe { core::mem::transmute(slice) }
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
&self.0
|
||||
}
|
||||
}
|
||||
|
||||
impl SliceLen for &Bytes {
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'i> Stream for &'i Bytes {
|
||||
type Token = u8;
|
||||
type Slice = &'i [u8];
|
||||
|
||||
type IterOffsets = Enumerate<Cloned<Iter<'i, u8>>>;
|
||||
|
||||
type Checkpoint = Checkpoint<Self, Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.iter().cloned().enumerate()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.len()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
if self.is_empty() {
|
||||
None
|
||||
} else {
|
||||
let token = self[0];
|
||||
*self = &self[1..];
|
||||
Some(token)
|
||||
}
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
if self.is_empty() {
|
||||
None
|
||||
} else {
|
||||
Some(self[0])
|
||||
}
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.iter().position(|b| predicate(*b))
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
if let Some(needed) = tokens.checked_sub(self.len()).and_then(NonZeroUsize::new) {
|
||||
Err(Needed::Size(needed))
|
||||
} else {
|
||||
Ok(tokens)
|
||||
}
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
let (slice, next) = self.0.split_at(offset);
|
||||
*self = Bytes::from_bytes(next);
|
||||
slice
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
#[cfg(debug_assertions)]
|
||||
self.peek_slice(offset);
|
||||
|
||||
// SAFETY: `Stream::next_slice_unchecked` requires `offset` to be in bounds
|
||||
let slice = unsafe { self.0.get_unchecked(..offset) };
|
||||
// SAFETY: `Stream::next_slice_unchecked` requires `offset` to be in bounds
|
||||
let next = unsafe { self.0.get_unchecked(offset..) };
|
||||
*self = Bytes::from_bytes(next);
|
||||
slice
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
&self[..offset]
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
#[cfg(debug_assertions)]
|
||||
self.peek_slice(offset);
|
||||
|
||||
// SAFETY: `Stream::next_slice_unchecked` requires `offset` to be in bounds
|
||||
let slice = unsafe { self.0.get_unchecked(..offset) };
|
||||
slice
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(*self)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
*self = checkpoint.inner;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<E> Recover<E> for &Bytes {
|
||||
#[inline(always)]
|
||||
fn record_err(
|
||||
&mut self,
|
||||
_token_start: &Self::Checkpoint,
|
||||
_err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
Err(err)
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl StreamIsPartial for &Bytes {
|
||||
type PartialState = ();
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
// Already complete
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, _state: Self::PartialState) {}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl Offset for &Bytes {
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, start: &Self) -> usize {
|
||||
self.as_bytes().offset_from(&start.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Offset<<&'a Bytes as Stream>::Checkpoint> for &'a Bytes {
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<&'a Bytes as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl AsBytes for &Bytes {
|
||||
#[inline(always)]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
(*self).as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a, T> Compare<T> for &'a Bytes
|
||||
where
|
||||
&'a [u8]: Compare<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn compare(&self, t: T) -> CompareResult {
|
||||
let bytes = (*self).as_bytes();
|
||||
bytes.compare(t)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'i, S> FindSlice<S> for &'i Bytes
|
||||
where
|
||||
&'i [u8]: FindSlice<S>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn find_slice(&self, substr: S) -> Option<core::ops::Range<usize>> {
|
||||
let bytes = (*self).as_bytes();
|
||||
let offset = bytes.find_slice(substr);
|
||||
offset
|
||||
}
|
||||
}
|
||||
|
||||
impl UpdateSlice for &Bytes {
|
||||
#[inline(always)]
|
||||
fn update_slice(self, inner: Self::Slice) -> Self {
|
||||
Bytes::new(inner)
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Display for Bytes {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
<Self as fmt::UpperHex>::fmt(self, f)
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::Debug for Bytes {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
<Self as fmt::UpperHex>::fmt(self, f)
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::LowerHex for Bytes {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
for byte in self.as_bytes() {
|
||||
write!(f, "{byte:0>2x}")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl fmt::UpperHex for Bytes {
|
||||
#[inline]
|
||||
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
|
||||
for (i, byte) in self.as_bytes().iter().enumerate() {
|
||||
if 0 < i {
|
||||
let absolute = (self.as_bytes().as_ptr() as usize) + i;
|
||||
if f.alternate() && absolute != 0 && absolute % 4 == 0 {
|
||||
write!(f, "_")?;
|
||||
}
|
||||
}
|
||||
write!(f, "{byte:0>2X}")?;
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Deref for Bytes {
|
||||
type Target = [u8];
|
||||
|
||||
#[inline]
|
||||
fn deref(&self) -> &[u8] {
|
||||
self.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<usize> for Bytes {
|
||||
type Output = u8;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, idx: usize) -> &u8 {
|
||||
&self.as_bytes()[idx]
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeFull> for Bytes {
|
||||
type Output = Bytes;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, _: ops::RangeFull) -> &Bytes {
|
||||
self
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::Range<usize>> for Bytes {
|
||||
type Output = Bytes;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::Range<usize>) -> &Bytes {
|
||||
Bytes::new(&self.as_bytes()[r.start..r.end])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeInclusive<usize>> for Bytes {
|
||||
type Output = Bytes;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeInclusive<usize>) -> &Bytes {
|
||||
Bytes::new(&self.as_bytes()[*r.start()..=*r.end()])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeFrom<usize>> for Bytes {
|
||||
type Output = Bytes;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeFrom<usize>) -> &Bytes {
|
||||
Bytes::new(&self.as_bytes()[r.start..])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeTo<usize>> for Bytes {
|
||||
type Output = Bytes;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeTo<usize>) -> &Bytes {
|
||||
Bytes::new(&self.as_bytes()[..r.end])
|
||||
}
|
||||
}
|
||||
|
||||
impl ops::Index<ops::RangeToInclusive<usize>> for Bytes {
|
||||
type Output = Bytes;
|
||||
|
||||
#[inline]
|
||||
fn index(&self, r: ops::RangeToInclusive<usize>) -> &Bytes {
|
||||
Bytes::new(&self.as_bytes()[..=r.end])
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<[u8]> for Bytes {
|
||||
#[inline]
|
||||
fn as_ref(&self) -> &[u8] {
|
||||
self.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<Bytes> for [u8] {
|
||||
#[inline]
|
||||
fn as_ref(&self) -> &Bytes {
|
||||
Bytes::new(self)
|
||||
}
|
||||
}
|
||||
|
||||
impl AsRef<Bytes> for str {
|
||||
#[inline]
|
||||
fn as_ref(&self) -> &Bytes {
|
||||
Bytes::new(self)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "alloc")]
|
||||
impl alloc::borrow::ToOwned for Bytes {
|
||||
type Owned = alloc::vec::Vec<u8>;
|
||||
|
||||
#[inline]
|
||||
fn to_owned(&self) -> Self::Owned {
|
||||
alloc::vec::Vec::from(self.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "alloc")]
|
||||
impl core::borrow::Borrow<Bytes> for alloc::vec::Vec<u8> {
|
||||
#[inline]
|
||||
fn borrow(&self) -> &Bytes {
|
||||
Bytes::from_bytes(self.as_slice())
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> Default for &'a Bytes {
|
||||
fn default() -> &'a Bytes {
|
||||
Bytes::new(b"")
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> From<&'a [u8]> for &'a Bytes {
|
||||
#[inline]
|
||||
fn from(s: &'a [u8]) -> &'a Bytes {
|
||||
Bytes::new(s)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> From<&'a Bytes> for &'a [u8] {
|
||||
#[inline]
|
||||
fn from(s: &'a Bytes) -> &'a [u8] {
|
||||
Bytes::as_bytes(s)
|
||||
}
|
||||
}
|
||||
|
||||
impl<'a> From<&'a str> for &'a Bytes {
|
||||
#[inline]
|
||||
fn from(s: &'a str) -> &'a Bytes {
|
||||
Bytes::new(s.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
impl Eq for Bytes {}
|
||||
|
||||
impl PartialEq<Bytes> for Bytes {
|
||||
#[inline]
|
||||
fn eq(&self, other: &Bytes) -> bool {
|
||||
self.as_bytes() == other.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl_partial_eq!(Bytes, [u8]);
|
||||
impl_partial_eq!(Bytes, &'a [u8]);
|
||||
impl_partial_eq!(Bytes, str);
|
||||
impl_partial_eq!(Bytes, &'a str);
|
||||
|
||||
impl PartialOrd for Bytes {
|
||||
#[inline]
|
||||
fn partial_cmp(&self, other: &Bytes) -> Option<Ordering> {
|
||||
Some(self.cmp(other))
|
||||
}
|
||||
}
|
||||
|
||||
impl Ord for Bytes {
|
||||
#[inline]
|
||||
fn cmp(&self, other: &Bytes) -> Ordering {
|
||||
Ord::cmp(self.as_bytes(), other.as_bytes())
|
||||
}
|
||||
}
|
||||
|
||||
impl_partial_ord!(Bytes, [u8]);
|
||||
impl_partial_ord!(Bytes, &'a [u8]);
|
||||
impl_partial_ord!(Bytes, str);
|
||||
impl_partial_ord!(Bytes, &'a str);
|
||||
|
||||
#[cfg(test)]
|
||||
mod test {
|
||||
use crate::stream::Bytes;
|
||||
|
||||
#[test]
|
||||
fn partial_eq_bytes_byte_slice() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = Bytes::new(input);
|
||||
assert!(actual == input);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_eq_byte_slice_bytes() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = Bytes::new(input);
|
||||
assert!(input == actual);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_eq_bytes_str() {
|
||||
let input = "foo";
|
||||
let actual = Bytes::new(input);
|
||||
assert!(actual == input);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_eq_str_bytes() {
|
||||
let input = "foo";
|
||||
let actual = Bytes::new(input);
|
||||
assert!(input == actual);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_bytes_byte_slice() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = Bytes::new(input);
|
||||
assert!(actual.partial_cmp(input) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_byte_slice_bytes() {
|
||||
let input = b"foo".as_slice();
|
||||
let actual = Bytes::new(input);
|
||||
assert!(input.partial_cmp(actual) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_bytes_str() {
|
||||
let input = "foo";
|
||||
let actual = Bytes::new(input);
|
||||
assert!(actual.partial_cmp(input) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn partial_ord_str_bytes() {
|
||||
let input = "foo";
|
||||
let actual = Bytes::new(input);
|
||||
assert!(input.partial_cmp(actual) == Some(core::cmp::Ordering::Equal));
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "std"))]
|
||||
mod display {
|
||||
use crate::stream::Bytes;
|
||||
|
||||
#[test]
|
||||
fn clean() {
|
||||
assert_eq!(&format!("{}", Bytes::new(b"abc")), "616263");
|
||||
assert_eq!(&format!("{}", Bytes::new(b"\xf0\x28\x8c\xbc")), "F0288CBC");
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(all(test, feature = "std"))]
|
||||
mod debug {
|
||||
use crate::stream::Bytes;
|
||||
use crate::stream::Stream as _;
|
||||
use snapbox::assert_data_eq;
|
||||
use snapbox::str;
|
||||
|
||||
#[test]
|
||||
fn test_debug() {
|
||||
let input = Bytes::new(b"\0\0\0 ftypisom\0\0\x02\0isomiso2avc1mp");
|
||||
let expected = str!["000000206674797069736F6D0000020069736F6D69736F32617663316D70"];
|
||||
assert_data_eq!(&format!("{input:?}"), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_pretty_debug() {
|
||||
let input = Bytes::new(b"\0\0\0 ftypisom\0\0\x02\0isomiso2avc1mp");
|
||||
let expected = str!["000000206674797069736F6D0000020069736F6D69736F32617663316D70"];
|
||||
assert_data_eq!(&format!("{input:#?}").replace('_', ""), expected);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_trace() {
|
||||
let input = Bytes::new(b"\0\0\0 ftypisom\0\0\x02\0isomiso2avc1mp");
|
||||
let expected = str!["000000206674797069736F6D0000020069736F6D69736F32617663316D70"];
|
||||
assert_data_eq!(
|
||||
crate::util::from_fn(|f| input.trace(f))
|
||||
.to_string()
|
||||
.replace('_', ""),
|
||||
expected
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_sliced() {
|
||||
// Output can change from run-to-run
|
||||
let total = Bytes::new(b"12345678901234567890");
|
||||
let _ = format!("{total:#?}");
|
||||
let _ = format!("{:#?}", &total[1..]);
|
||||
let _ = format!("{:#?}", &total[10..]);
|
||||
}
|
||||
}
|
||||
+326
@@ -0,0 +1,326 @@
|
||||
use crate::error::Needed;
|
||||
use crate::stream::AsBStr;
|
||||
use crate::stream::AsBytes;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::FindSlice;
|
||||
use crate::stream::Location;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
|
||||
/// Allow collecting the span of a parsed token within a slice
|
||||
///
|
||||
/// Converting byte offsets to line or column numbers is left up to the user, as computing column
|
||||
/// numbers requires domain knowledge (are columns byte-based, codepoint-based, or grapheme-based?)
|
||||
/// and O(n) iteration over the input to determine codepoint and line boundaries.
|
||||
///
|
||||
/// [The `line-span` crate](https://docs.rs/line-span/latest/line_span/) can help with converting
|
||||
/// byte offsets to line numbers.
|
||||
///
|
||||
/// See [`Parser::span`][crate::Parser::span] and [`Parser::with_span`][crate::Parser::with_span] for more details
|
||||
#[derive(Copy, Clone, Default, PartialEq, Eq, PartialOrd, Ord)]
|
||||
#[doc(alias = "LocatingSliceSpan")]
|
||||
#[doc(alias = "Located")]
|
||||
pub struct LocatingSlice<I> {
|
||||
initial: I,
|
||||
input: I,
|
||||
}
|
||||
|
||||
impl<I> LocatingSlice<I>
|
||||
where
|
||||
I: Clone + Offset,
|
||||
{
|
||||
/// Wrap another Stream with span tracking
|
||||
pub fn new(input: I) -> Self {
|
||||
let initial = input.clone();
|
||||
Self { initial, input }
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
// Assumptions:
|
||||
// - Index offsets is sufficient
|
||||
// - Tokens are continuous
|
||||
self.input.offset_from(&self.initial)
|
||||
}
|
||||
#[inline]
|
||||
fn current_token_start(&self) -> usize {
|
||||
// Assumptions:
|
||||
// - Index offsets is sufficient
|
||||
self.input.offset_from(&self.initial)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> LocatingSlice<I>
|
||||
where
|
||||
I: Clone + Stream + Offset,
|
||||
{
|
||||
/// Reset the stream to the start
|
||||
///
|
||||
/// This is useful for formats that encode a graph with addresses relative to the start of the
|
||||
/// input.
|
||||
#[doc(alias = "fseek")]
|
||||
#[inline]
|
||||
pub fn reset_to_start(&mut self) {
|
||||
let start = self.initial.checkpoint();
|
||||
self.input.reset(&start);
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> AsRef<I> for LocatingSlice<I> {
|
||||
#[inline(always)]
|
||||
fn as_ref(&self) -> &I {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: core::fmt::Debug> core::fmt::Debug for LocatingSlice<I> {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> core::ops::Deref for LocatingSlice<I> {
|
||||
type Target = I;
|
||||
|
||||
#[inline(always)]
|
||||
fn deref(&self) -> &Self::Target {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: core::fmt::Display> core::fmt::Display for LocatingSlice<I> {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> SliceLen for LocatingSlice<I>
|
||||
where
|
||||
I: SliceLen,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.input.slice_len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream> Stream for LocatingSlice<I> {
|
||||
type Token = <I as Stream>::Token;
|
||||
type Slice = <I as Stream>::Slice;
|
||||
|
||||
type IterOffsets = <I as Stream>::IterOffsets;
|
||||
|
||||
type Checkpoint = Checkpoint<I::Checkpoint, Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.input.iter_offsets()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.input.eof_offset()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
self.input.next_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
self.input.peek_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.input.offset_for(predicate)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
self.input.offset_at(tokens)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
self.input.next_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.next_slice_unchecked(offset) }
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
self.input.peek_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.peek_slice_unchecked(offset) }
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(self.input.checkpoint())
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
self.input.reset(&checkpoint.inner);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
#![allow(deprecated)]
|
||||
self.input.raw()
|
||||
}
|
||||
|
||||
fn trace(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.trace(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Location for LocatingSlice<I>
|
||||
where
|
||||
I: Clone + Offset,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
self.previous_token_end()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> usize {
|
||||
self.current_token_start()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<I, E> Recover<E> for LocatingSlice<I>
|
||||
where
|
||||
I: Recover<E>,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn record_err(
|
||||
&mut self,
|
||||
_token_start: &Self::Checkpoint,
|
||||
_err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
Err(err)
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> StreamIsPartial for LocatingSlice<I>
|
||||
where
|
||||
I: StreamIsPartial,
|
||||
{
|
||||
type PartialState = I::PartialState;
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
self.input.complete()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, state: Self::PartialState) {
|
||||
self.input.restore_partial(state);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
I::is_partial_supported()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial(&self) -> bool {
|
||||
self.input.is_partial()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Offset for LocatingSlice<I>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &Self) -> usize {
|
||||
self.offset_from(&other.checkpoint())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Offset<<LocatingSlice<I> as Stream>::Checkpoint> for LocatingSlice<I>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<LocatingSlice<I> as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> AsBytes for LocatingSlice<I>
|
||||
where
|
||||
I: AsBytes,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
self.input.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> AsBStr for LocatingSlice<I>
|
||||
where
|
||||
I: AsBStr,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bstr(&self) -> &[u8] {
|
||||
self.input.as_bstr()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, U> Compare<U> for LocatingSlice<I>
|
||||
where
|
||||
I: Compare<U>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn compare(&self, other: U) -> CompareResult {
|
||||
self.input.compare(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, T> FindSlice<T> for LocatingSlice<I>
|
||||
where
|
||||
I: FindSlice<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn find_slice(&self, substr: T) -> Option<core::ops::Range<usize>> {
|
||||
self.input.find_slice(substr)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> UpdateSlice for LocatingSlice<I>
|
||||
where
|
||||
I: UpdateSlice,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn update_slice(mut self, inner: Self::Slice) -> Self {
|
||||
self.input = I::update_slice(self.input, inner);
|
||||
self
|
||||
}
|
||||
}
|
||||
+2241
File diff suppressed because it is too large.
Load diff
+354
@@ -0,0 +1,354 @@
|
||||
use crate::error::Needed;
|
||||
use crate::stream::AsBStr;
|
||||
use crate::stream::AsBytes;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::FindSlice;
|
||||
use crate::stream::Location;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
|
||||
/// Mark the input as a partial buffer for streaming input.
|
||||
///
|
||||
/// Complete input means that we already have all of the data. This will be the common case with
|
||||
/// small files that can be read entirely to memory.
|
||||
///
|
||||
/// In contrast, streaming input assumes that we might not have all of the data.
|
||||
/// This can happen with some network protocol or large file parsers, where the
|
||||
/// input buffer can be full and need to be resized or refilled.
|
||||
/// - [`ErrMode::Incomplete`][crate::error::ErrMode::Incomplete] will report how much more data is needed.
|
||||
/// - [`Parser::complete_err`][crate::Parser::complete_err] transform
|
||||
/// [`ErrMode::Incomplete`][crate::error::ErrMode::Incomplete] to
|
||||
/// [`ErrMode::Backtrack`][crate::error::ErrMode::Backtrack]
|
||||
///
|
||||
/// See also [`StreamIsPartial`] to tell whether the input supports complete or partial parsing.
|
||||
///
|
||||
/// See also [Special Topics: Parsing Partial Input][crate::_topic::partial].
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// Here is how it works in practice:
|
||||
///
|
||||
/// ```rust
|
||||
/// # use winnow::{Result, error::ErrMode, error::Needed, error::ContextError, token, ascii, stream::Partial};
|
||||
/// # use winnow::prelude::*;
|
||||
///
|
||||
/// fn take_partial<'s>(i: &mut Partial<&'s [u8]>) -> ModalResult<&'s [u8], ContextError> {
|
||||
/// token::take(4u8).parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// fn take_complete<'s>(i: &mut &'s [u8]) -> ModalResult<&'s [u8], ContextError> {
|
||||
/// token::take(4u8).parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// // both parsers will take 4 bytes as expected
|
||||
/// assert_eq!(take_partial.parse_peek(Partial::new(&b"abcde"[..])), Ok((Partial::new(&b"e"[..]), &b"abcd"[..])));
|
||||
/// assert_eq!(take_complete.parse_peek(&b"abcde"[..]), Ok((&b"e"[..], &b"abcd"[..])));
|
||||
///
|
||||
/// // if the input is smaller than 4 bytes, the partial parser
|
||||
/// // will return `Incomplete` to indicate that we need more data
|
||||
/// assert_eq!(take_partial.parse_peek(Partial::new(&b"abc"[..])), Err(ErrMode::Incomplete(Needed::new(1))));
|
||||
///
|
||||
/// // but the complete parser will return an error
|
||||
/// assert!(take_complete.parse_peek(&b"abc"[..]).is_err());
|
||||
///
|
||||
/// // the alpha0 function takes 0 or more alphabetic characters
|
||||
/// fn alpha0_partial<'s>(i: &mut Partial<&'s str>) -> ModalResult<&'s str, ContextError> {
|
||||
/// ascii::alpha0.parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// fn alpha0_complete<'s>(i: &mut &'s str) -> ModalResult<&'s str, ContextError> {
|
||||
/// ascii::alpha0.parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// // if there's a clear limit to the taken characters, both parsers work the same way
|
||||
/// assert_eq!(alpha0_partial.parse_peek(Partial::new("abcd;")), Ok((Partial::new(";"), "abcd")));
|
||||
/// assert_eq!(alpha0_complete.parse_peek("abcd;"), Ok((";", "abcd")));
|
||||
///
|
||||
/// // but when there's no limit, the partial version returns `Incomplete`, because it cannot
|
||||
/// // know if more input data should be taken. The whole input could be "abcd;", or
|
||||
/// // "abcde;"
|
||||
/// assert_eq!(alpha0_partial.parse_peek(Partial::new("abcd")), Err(ErrMode::Incomplete(Needed::new(1))));
|
||||
///
|
||||
/// // while the complete version knows that all of the data is there
|
||||
/// assert_eq!(alpha0_complete.parse_peek("abcd"), Ok(("", "abcd")));
|
||||
/// ```
|
||||
#[derive(Copy, Clone, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
pub struct Partial<I> {
|
||||
input: I,
|
||||
partial: bool,
|
||||
}
|
||||
|
||||
impl<I> Partial<I>
|
||||
where
|
||||
I: StreamIsPartial,
|
||||
{
|
||||
/// Create a partial input
|
||||
#[inline]
|
||||
pub fn new(input: I) -> Self {
|
||||
debug_assert!(
|
||||
!I::is_partial_supported(),
|
||||
"`Partial` can only wrap complete sources"
|
||||
);
|
||||
let partial = true;
|
||||
Self { input, partial }
|
||||
}
|
||||
|
||||
/// Extract the original [`Stream`]
|
||||
#[inline(always)]
|
||||
pub fn into_inner(self) -> I {
|
||||
self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Default for Partial<I>
|
||||
where
|
||||
I: Default + StreamIsPartial,
|
||||
{
|
||||
#[inline]
|
||||
fn default() -> Self {
|
||||
Self::new(I::default())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> core::ops::Deref for Partial<I> {
|
||||
type Target = I;
|
||||
|
||||
#[inline(always)]
|
||||
fn deref(&self) -> &Self::Target {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: core::fmt::Display> core::fmt::Display for Partial<I> {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> SliceLen for Partial<I>
|
||||
where
|
||||
I: SliceLen,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.input.slice_len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream> Stream for Partial<I> {
|
||||
type Token = <I as Stream>::Token;
|
||||
type Slice = <I as Stream>::Slice;
|
||||
|
||||
type IterOffsets = <I as Stream>::IterOffsets;
|
||||
|
||||
type Checkpoint = Checkpoint<I::Checkpoint, Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.input.iter_offsets()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.input.eof_offset()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
self.input.next_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
self.input.peek_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.input.offset_for(predicate)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
self.input.offset_at(tokens)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
self.input.next_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.next_slice_unchecked(offset) }
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
self.input.peek_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.peek_slice_unchecked(offset) }
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(self.input.checkpoint())
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
self.input.reset(&checkpoint.inner);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Location for Partial<I>
|
||||
where
|
||||
I: Location,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
self.input.previous_token_end()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> usize {
|
||||
self.input.current_token_start()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<I, E> Recover<E> for Partial<I>
|
||||
where
|
||||
I: Recover<E>,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn record_err(
|
||||
&mut self,
|
||||
_token_start: &Self::Checkpoint,
|
||||
_err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
Err(err)
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> StreamIsPartial for Partial<I>
|
||||
where
|
||||
I: StreamIsPartial,
|
||||
{
|
||||
type PartialState = bool;
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
core::mem::replace(&mut self.partial, false)
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, state: Self::PartialState) {
|
||||
self.partial = state;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial(&self) -> bool {
|
||||
self.partial
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Offset for Partial<I>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, start: &Self) -> usize {
|
||||
self.offset_from(&start.checkpoint())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> Offset<<Partial<I> as Stream>::Checkpoint> for Partial<I>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<Partial<I> as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> AsBytes for Partial<I>
|
||||
where
|
||||
I: AsBytes,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
self.input.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> AsBStr for Partial<I>
|
||||
where
|
||||
I: AsBStr,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bstr(&self) -> &[u8] {
|
||||
self.input.as_bstr()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, T> Compare<T> for Partial<I>
|
||||
where
|
||||
I: Compare<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn compare(&self, t: T) -> CompareResult {
|
||||
self.input.compare(t)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, T> FindSlice<T> for Partial<I>
|
||||
where
|
||||
I: FindSlice<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn find_slice(&self, substr: T) -> Option<core::ops::Range<usize>> {
|
||||
self.input.find_slice(substr)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I> UpdateSlice for Partial<I>
|
||||
where
|
||||
I: UpdateSlice,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn update_slice(self, inner: Self::Slice) -> Self {
|
||||
Partial {
|
||||
input: I::update_slice(self.input, inner),
|
||||
partial: self.partial,
|
||||
}
|
||||
}
|
||||
}
|
||||
+143
@@ -0,0 +1,143 @@
|
||||
/// A range bounded inclusively for counting parses performed
|
||||
///
|
||||
/// This is flexible in what can be converted to a [Range]:
|
||||
/// ```rust
|
||||
/// # #[cfg(feature = "std")] {
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::token::any;
|
||||
/// # use winnow::combinator::repeat;
|
||||
/// # fn inner(input: &mut &str) -> ModalResult<char> {
|
||||
/// # any.parse_next(input)
|
||||
/// # }
|
||||
/// # let mut input = "0123456789012345678901234567890123456789";
|
||||
/// # let input = &mut input;
|
||||
/// let parser: Vec<_> = repeat(5, inner).parse_next(input).unwrap();
|
||||
/// # let mut input = "0123456789012345678901234567890123456789";
|
||||
/// # let input = &mut input;
|
||||
/// let parser: Vec<_> = repeat(.., inner).parse_next(input).unwrap();
|
||||
/// # let mut input = "0123456789012345678901234567890123456789";
|
||||
/// # let input = &mut input;
|
||||
/// let parser: Vec<_> = repeat(1.., inner).parse_next(input).unwrap();
|
||||
/// # let mut input = "0123456789012345678901234567890123456789";
|
||||
/// # let input = &mut input;
|
||||
/// let parser: Vec<_> = repeat(5..8, inner).parse_next(input).unwrap();
|
||||
/// # let mut input = "0123456789012345678901234567890123456789";
|
||||
/// # let input = &mut input;
|
||||
/// let parser: Vec<_> = repeat(5..=8, inner).parse_next(input).unwrap();
|
||||
/// # }
|
||||
/// ```
|
||||
#[derive(PartialEq, Eq, Copy, Clone)]
|
||||
pub struct Range {
|
||||
pub(crate) start_inclusive: usize,
|
||||
pub(crate) end_inclusive: Option<usize>,
|
||||
}
|
||||
|
||||
impl Range {
|
||||
#[inline(always)]
|
||||
fn raw(start_inclusive: usize, end_inclusive: Option<usize>) -> Self {
|
||||
Self {
|
||||
start_inclusive,
|
||||
end_inclusive,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl core::ops::RangeBounds<usize> for Range {
|
||||
#[inline(always)]
|
||||
fn start_bound(&self) -> core::ops::Bound<&usize> {
|
||||
core::ops::Bound::Included(&self.start_inclusive)
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn end_bound(&self) -> core::ops::Bound<&usize> {
|
||||
if let Some(end_inclusive) = &self.end_inclusive {
|
||||
core::ops::Bound::Included(end_inclusive)
|
||||
} else {
|
||||
core::ops::Bound::Unbounded
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl From<usize> for Range {
|
||||
#[inline(always)]
|
||||
fn from(fixed: usize) -> Self {
|
||||
(fixed..=fixed).into()
|
||||
}
|
||||
}
|
||||
|
||||
impl From<core::ops::Range<usize>> for Range {
|
||||
#[inline(always)]
|
||||
fn from(range: core::ops::Range<usize>) -> Self {
|
||||
let start_inclusive = range.start;
|
||||
let end_inclusive = Some(range.end.saturating_sub(1));
|
||||
Self::raw(start_inclusive, end_inclusive)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<core::ops::RangeFull> for Range {
|
||||
#[inline(always)]
|
||||
fn from(_: core::ops::RangeFull) -> Self {
|
||||
let start_inclusive = 0;
|
||||
let end_inclusive = None;
|
||||
Self::raw(start_inclusive, end_inclusive)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<core::ops::RangeFrom<usize>> for Range {
|
||||
#[inline(always)]
|
||||
fn from(range: core::ops::RangeFrom<usize>) -> Self {
|
||||
let start_inclusive = range.start;
|
||||
let end_inclusive = None;
|
||||
Self::raw(start_inclusive, end_inclusive)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<core::ops::RangeTo<usize>> for Range {
|
||||
#[inline(always)]
|
||||
fn from(range: core::ops::RangeTo<usize>) -> Self {
|
||||
let start_inclusive = 0;
|
||||
let end_inclusive = Some(range.end.saturating_sub(1));
|
||||
Self::raw(start_inclusive, end_inclusive)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<core::ops::RangeInclusive<usize>> for Range {
|
||||
#[inline(always)]
|
||||
fn from(range: core::ops::RangeInclusive<usize>) -> Self {
|
||||
let start_inclusive = *range.start();
|
||||
let end_inclusive = Some(*range.end());
|
||||
Self::raw(start_inclusive, end_inclusive)
|
||||
}
|
||||
}
|
||||
|
||||
impl From<core::ops::RangeToInclusive<usize>> for Range {
|
||||
#[inline(always)]
|
||||
fn from(range: core::ops::RangeToInclusive<usize>) -> Self {
|
||||
let start_inclusive = 0;
|
||||
let end_inclusive = Some(range.end);
|
||||
Self::raw(start_inclusive, end_inclusive)
|
||||
}
|
||||
}
|
||||
|
||||
impl core::fmt::Display for Range {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.start_inclusive.fmt(f)?;
|
||||
match self.end_inclusive {
|
||||
Some(e) if e == self.start_inclusive => {}
|
||||
Some(e) => {
|
||||
"..=".fmt(f)?;
|
||||
e.fmt(f)?;
|
||||
}
|
||||
None => {
|
||||
"..".fmt(f)?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
|
||||
impl core::fmt::Debug for Range {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
write!(f, "{self}")
|
||||
}
|
||||
}
|
||||
+344
@@ -0,0 +1,344 @@
|
||||
use crate::error::FromRecoverableError;
|
||||
use crate::error::Needed;
|
||||
use crate::stream::AsBStr;
|
||||
use crate::stream::AsBytes;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::FindSlice;
|
||||
use crate::stream::Location;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
|
||||
/// Allow recovering from parse errors, capturing them as the parser continues
|
||||
///
|
||||
/// Generally, this will be used indirectly via
|
||||
/// [`RecoverableParser::recoverable_parse`][crate::RecoverableParser::recoverable_parse].
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
input: I,
|
||||
errors: Vec<E>,
|
||||
is_recoverable: bool,
|
||||
}
|
||||
|
||||
impl<I, E> Default for Recoverable<I, E>
|
||||
where
|
||||
I: Default + Stream,
|
||||
{
|
||||
#[inline]
|
||||
fn default() -> Self {
|
||||
Self::new(I::default())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
/// Track recoverable errors with the stream
|
||||
#[inline]
|
||||
pub fn new(input: I) -> Self {
|
||||
Self {
|
||||
input,
|
||||
errors: Default::default(),
|
||||
is_recoverable: true,
|
||||
}
|
||||
}
|
||||
|
||||
/// Act as a normal stream
|
||||
#[inline]
|
||||
pub fn unrecoverable(input: I) -> Self {
|
||||
Self {
|
||||
input,
|
||||
errors: Default::default(),
|
||||
is_recoverable: false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Access the current input and errors
|
||||
#[inline]
|
||||
pub fn into_parts(self) -> (I, Vec<E>) {
|
||||
(self.input, self.errors)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> AsRef<I> for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_ref(&self) -> &I {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> core::ops::Deref for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
type Target = I;
|
||||
|
||||
#[inline(always)]
|
||||
fn deref(&self) -> &Self::Target {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: core::fmt::Display, E> core::fmt::Display for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
core::fmt::Display::fmt(&self.input, f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> SliceLen for Recoverable<I, E>
|
||||
where
|
||||
I: SliceLen,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.input.slice_len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E: core::fmt::Debug> Stream for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
{
|
||||
type Token = <I as Stream>::Token;
|
||||
type Slice = <I as Stream>::Slice;
|
||||
|
||||
type IterOffsets = <I as Stream>::IterOffsets;
|
||||
|
||||
type Checkpoint = Checkpoint<I::Checkpoint, Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.input.iter_offsets()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.input.eof_offset()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
self.input.next_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
self.input.peek_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.input.offset_for(predicate)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
self.input.offset_at(tokens)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
self.input.next_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.next_slice_unchecked(offset) }
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
self.input.peek_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.peek_slice_unchecked(offset) }
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(self.input.checkpoint())
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
self.input.reset(&checkpoint.inner);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> Location for Recoverable<I, E>
|
||||
where
|
||||
I: Location,
|
||||
I: Stream,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
self.input.previous_token_end()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> usize {
|
||||
self.input.current_token_start()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E, R> Recover<E> for Recoverable<I, R>
|
||||
where
|
||||
I: Stream,
|
||||
R: FromRecoverableError<Self, E>,
|
||||
R: core::fmt::Debug,
|
||||
E: crate::error::ParserError<Self>,
|
||||
{
|
||||
fn record_err(
|
||||
&mut self,
|
||||
token_start: &Self::Checkpoint,
|
||||
err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
if self.is_recoverable {
|
||||
if err.is_incomplete() {
|
||||
Err(err)
|
||||
} else {
|
||||
self.errors
|
||||
.push(R::from_recoverable_error(token_start, err_start, self, err));
|
||||
Ok(())
|
||||
}
|
||||
} else {
|
||||
Err(err)
|
||||
}
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
true
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> StreamIsPartial for Recoverable<I, E>
|
||||
where
|
||||
I: StreamIsPartial,
|
||||
I: Stream,
|
||||
{
|
||||
type PartialState = I::PartialState;
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
self.input.complete()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, state: Self::PartialState) {
|
||||
self.input.restore_partial(state);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
I::is_partial_supported()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial(&self) -> bool {
|
||||
self.input.is_partial()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> Offset for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
E: core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &Self) -> usize {
|
||||
self.offset_from(&other.checkpoint())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> Offset<<Recoverable<I, E> as Stream>::Checkpoint> for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
E: core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<Recoverable<I, E> as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> AsBytes for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
I: AsBytes,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
self.input.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> AsBStr for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
I: AsBStr,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bstr(&self) -> &[u8] {
|
||||
self.input.as_bstr()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E, U> Compare<U> for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
I: Compare<U>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn compare(&self, other: U) -> CompareResult {
|
||||
self.input.compare(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E, T> FindSlice<T> for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
I: FindSlice<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn find_slice(&self, substr: T) -> Option<core::ops::Range<usize>> {
|
||||
self.input.find_slice(substr)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, E> UpdateSlice for Recoverable<I, E>
|
||||
where
|
||||
I: Stream,
|
||||
I: UpdateSlice,
|
||||
E: core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn update_slice(mut self, inner: Self::Slice) -> Self {
|
||||
self.input = I::update_slice(self.input, inner);
|
||||
self
|
||||
}
|
||||
}
|
||||
+312
@@ -0,0 +1,312 @@
|
||||
use crate::error::Needed;
|
||||
use crate::stream::AsBStr;
|
||||
use crate::stream::AsBytes;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::FindSlice;
|
||||
use crate::stream::Location;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
|
||||
/// Thread global state through your parsers
|
||||
///
|
||||
/// Use cases
|
||||
/// - Recursion checks
|
||||
/// - Error recovery
|
||||
/// - Debugging
|
||||
///
|
||||
/// # Example
|
||||
///
|
||||
/// ```
|
||||
/// # use std::cell::Cell;
|
||||
/// # use winnow::prelude::*;
|
||||
/// # use winnow::stream::Stateful;
|
||||
/// # use winnow::ascii::alpha1;
|
||||
/// # type Error = ();
|
||||
///
|
||||
/// #[derive(Debug)]
|
||||
/// struct State<'s>(&'s mut u32);
|
||||
///
|
||||
/// impl<'s> State<'s> {
|
||||
/// fn count(&mut self) {
|
||||
/// *self.0 += 1;
|
||||
/// }
|
||||
/// }
|
||||
///
|
||||
/// type Stream<'is> = Stateful<&'is str, State<'is>>;
|
||||
///
|
||||
/// fn word<'s>(i: &mut Stream<'s>) -> ModalResult<&'s str> {
|
||||
/// i.state.count();
|
||||
/// alpha1.parse_next(i)
|
||||
/// }
|
||||
///
|
||||
/// let data = "Hello";
|
||||
/// let mut state = 0;
|
||||
/// let input = Stream { input: data, state: State(&mut state) };
|
||||
/// let output = word.parse(input).unwrap();
|
||||
/// assert_eq!(state, 1);
|
||||
/// ```
|
||||
#[derive(Clone, Copy, Debug, Default, Eq, PartialEq)]
|
||||
#[doc(alias = "LocatingSliceSpan")]
|
||||
pub struct Stateful<I, S> {
|
||||
/// Inner input being wrapped in state
|
||||
pub input: I,
|
||||
/// User-provided state
|
||||
pub state: S,
|
||||
}
|
||||
|
||||
impl<I, S> AsRef<I> for Stateful<I, S> {
|
||||
#[inline(always)]
|
||||
fn as_ref(&self) -> &I {
|
||||
&self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> core::ops::Deref for Stateful<I, S> {
|
||||
type Target = I;
|
||||
|
||||
#[inline(always)]
|
||||
fn deref(&self) -> &Self::Target {
|
||||
self.as_ref()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: core::fmt::Display, S> core::fmt::Display for Stateful<I, S> {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> SliceLen for Stateful<I, S>
|
||||
where
|
||||
I: SliceLen,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.input.slice_len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I: Stream, S: core::fmt::Debug> Stream for Stateful<I, S> {
|
||||
type Token = <I as Stream>::Token;
|
||||
type Slice = <I as Stream>::Slice;
|
||||
|
||||
type IterOffsets = <I as Stream>::IterOffsets;
|
||||
|
||||
type Checkpoint = Checkpoint<I::Checkpoint, Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.input.iter_offsets()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.input.eof_offset()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
self.input.next_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
self.input.peek_token()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.input.offset_for(predicate)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
self.input.offset_at(tokens)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
self.input.next_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.next_slice_unchecked(offset) }
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
self.input.peek_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.peek_slice_unchecked(offset) }
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(self.input.checkpoint())
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
self.input.reset(&checkpoint.inner);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
#![allow(deprecated)]
|
||||
self.input.raw()
|
||||
}
|
||||
|
||||
fn trace(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.trace(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> Location for Stateful<I, S>
|
||||
where
|
||||
I: Location,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
self.input.previous_token_end()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> usize {
|
||||
self.input.current_token_start()
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<I, E, S> Recover<E> for Stateful<I, S>
|
||||
where
|
||||
I: Recover<E>,
|
||||
I: Stream,
|
||||
S: Clone + core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn record_err(
|
||||
&mut self,
|
||||
_token_start: &Self::Checkpoint,
|
||||
_err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
Err(err)
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> StreamIsPartial for Stateful<I, S>
|
||||
where
|
||||
I: StreamIsPartial,
|
||||
{
|
||||
type PartialState = I::PartialState;
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
self.input.complete()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, state: Self::PartialState) {
|
||||
self.input.restore_partial(state);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
I::is_partial_supported()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial(&self) -> bool {
|
||||
self.input.is_partial()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> Offset for Stateful<I, S>
|
||||
where
|
||||
I: Stream,
|
||||
S: Clone + core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, start: &Self) -> usize {
|
||||
self.offset_from(&start.checkpoint())
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> Offset<<Stateful<I, S> as Stream>::Checkpoint> for Stateful<I, S>
|
||||
where
|
||||
I: Stream,
|
||||
S: core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<Stateful<I, S> as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> AsBytes for Stateful<I, S>
|
||||
where
|
||||
I: AsBytes,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bytes(&self) -> &[u8] {
|
||||
self.input.as_bytes()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> AsBStr for Stateful<I, S>
|
||||
where
|
||||
I: AsBStr,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn as_bstr(&self) -> &[u8] {
|
||||
self.input.as_bstr()
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S, U> Compare<U> for Stateful<I, S>
|
||||
where
|
||||
I: Compare<U>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn compare(&self, other: U) -> CompareResult {
|
||||
self.input.compare(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S, T> FindSlice<T> for Stateful<I, S>
|
||||
where
|
||||
I: FindSlice<T>,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn find_slice(&self, substr: T) -> Option<core::ops::Range<usize>> {
|
||||
self.input.find_slice(substr)
|
||||
}
|
||||
}
|
||||
|
||||
impl<I, S> UpdateSlice for Stateful<I, S>
|
||||
where
|
||||
I: UpdateSlice,
|
||||
S: Clone + core::fmt::Debug,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn update_slice(mut self, inner: Self::Slice) -> Self {
|
||||
self.input = I::update_slice(self.input, inner);
|
||||
self
|
||||
}
|
||||
}
|
||||
+275
@@ -0,0 +1,275 @@
|
||||
#[cfg(feature = "std")]
|
||||
use proptest::prelude::*;
|
||||
|
||||
use crate::error::ErrMode;
|
||||
use crate::error::ErrMode::Backtrack;
|
||||
use crate::error::InputError;
|
||||
use crate::token::literal;
|
||||
use crate::{
|
||||
combinator::{separated, separated_pair},
|
||||
ModalResult, Parser,
|
||||
};
|
||||
|
||||
use super::*;
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
#[test]
|
||||
fn test_fxhashmap_compiles() {
|
||||
let input = "a=b";
|
||||
fn pair(i: &mut &str) -> ModalResult<(char, char)> {
|
||||
let out = separated_pair('a', '=', 'b').parse_next(i)?;
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
let _: rustc_hash::FxHashMap<char, char> = separated(0.., pair, ',').parse(input).unwrap();
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_offset_u8() {
|
||||
let s = b"abcd123";
|
||||
let a = &s[..];
|
||||
let b = &a[2..];
|
||||
let c = &a[..4];
|
||||
let d = &a[3..5];
|
||||
assert_eq!(b.offset_from(&a), 2);
|
||||
assert_eq!(c.offset_from(&a), 0);
|
||||
assert_eq!(d.offset_from(&a), 3);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_offset_str() {
|
||||
let a = "abcřèÂßÇd123";
|
||||
let b = &a[7..];
|
||||
let c = &a[..5];
|
||||
let d = &a[5..9];
|
||||
assert_eq!(b.offset_from(&a), 7);
|
||||
assert_eq!(c.offset_from(&a), 0);
|
||||
assert_eq!(d.offset_from(&a), 5);
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg(feature = "alloc")]
|
||||
fn test_bit_stream_empty() {
|
||||
let i = (&b""[..], 0);
|
||||
|
||||
let actual = i.iter_offsets().collect::<alloc::vec::Vec<_>>();
|
||||
assert_eq!(actual, vec![]);
|
||||
|
||||
let actual = i.eof_offset();
|
||||
assert_eq!(actual, 0);
|
||||
|
||||
let actual = i.peek_token();
|
||||
assert_eq!(actual, None);
|
||||
|
||||
let actual = i.offset_for(|b| b);
|
||||
assert_eq!(actual, None);
|
||||
|
||||
let actual = i.offset_at(1);
|
||||
assert_eq!(actual, Err(Needed::new(1)));
|
||||
|
||||
let actual_slice = i.peek_slice(0);
|
||||
assert_eq!(actual_slice, (&b""[..], 0, 0));
|
||||
}
|
||||
|
||||
#[test]
|
||||
#[cfg(feature = "alloc")]
|
||||
fn test_bit_offset_empty() {
|
||||
let i = (&b""[..], 0);
|
||||
|
||||
let actual = i.offset_from(&i);
|
||||
assert_eq!(actual, 0);
|
||||
}
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
proptest! {
|
||||
#[test]
|
||||
#[cfg_attr(miri, ignore)] // See https://github.com/AltSysrq/proptest/issues/253
|
||||
fn bit_stream(byte_len in 0..20usize, start in 0..160usize) {
|
||||
bit_stream_inner(byte_len, start);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "std")]
|
||||
fn bit_stream_inner(byte_len: usize, start: usize) {
|
||||
let start = start.min(byte_len * 8);
|
||||
let start_byte = start / 8;
|
||||
let start_bit = start % 8;
|
||||
|
||||
let bytes = vec![0b1010_1010; byte_len];
|
||||
let i = (&bytes[start_byte..], start_bit);
|
||||
|
||||
let mut curr_i = i;
|
||||
let mut curr_offset = 0;
|
||||
while let Some(_token) = curr_i.peek_token() {
|
||||
let to_offset = curr_i.offset_from(&i);
|
||||
assert_eq!(curr_offset, to_offset);
|
||||
|
||||
let actual_slice = i.peek_slice(curr_offset);
|
||||
let expected_slice = i.clone().peek_slice(curr_offset);
|
||||
assert_eq!(actual_slice, expected_slice);
|
||||
|
||||
let at_offset = i.offset_at(curr_offset).unwrap();
|
||||
assert_eq!(curr_offset, at_offset);
|
||||
|
||||
let eof_offset = curr_i.eof_offset();
|
||||
let eof_slice = curr_i.peek_slice(eof_offset);
|
||||
let eof_slice_i = (eof_slice.0, eof_slice.1);
|
||||
assert_eq!(eof_slice_i, curr_i);
|
||||
|
||||
curr_offset += 1;
|
||||
let _ = curr_i.next_token();
|
||||
}
|
||||
assert_eq!(i.eof_offset(), curr_offset);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_partial_complete() {
|
||||
let mut i = Partial::new(&b""[..]);
|
||||
assert!(Partial::<&[u8]>::is_partial_supported());
|
||||
|
||||
assert!(i.is_partial(), "incomplete by default");
|
||||
let incomplete_state = i.complete();
|
||||
assert!(!i.is_partial(), "the stream should be marked as complete");
|
||||
|
||||
i.restore_partial(incomplete_state);
|
||||
assert!(i.is_partial(), "incomplete stream state should be restored");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_custom_slice() {
|
||||
type Token = usize;
|
||||
type TokenSlice<'i> = &'i [Token];
|
||||
|
||||
let mut tokens: TokenSlice<'_> = &[1, 2, 3, 4];
|
||||
|
||||
let input = &mut tokens;
|
||||
let start = input.checkpoint();
|
||||
let _ = input.next_token();
|
||||
let _ = input.next_token();
|
||||
let offset = input.offset_from(&start);
|
||||
assert_eq!(offset, 2);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_literal_support_char() {
|
||||
assert_eq!(
|
||||
literal::<_, _, ErrMode<InputError<_>>>('π').parse_peek("π"),
|
||||
Ok(("", "π"))
|
||||
);
|
||||
assert_eq!(
|
||||
literal::<_, _, ErrMode<InputError<_>>>('π').parse_peek("π3.14"),
|
||||
Ok(("3.14", "π"))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, _, ErrMode<InputError<_>>>("π").parse_peek("π3.14"),
|
||||
Ok(("3.14", "π"))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, _, ErrMode<InputError<_>>>('-').parse_peek("π"),
|
||||
Err(Backtrack(InputError::at("π")))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, Partial<&[u8]>, ErrMode<InputError<_>>>('π')
|
||||
.parse_peek(Partial::new(b"\xCF\x80")),
|
||||
Ok((Partial::new(Default::default()), "π".as_bytes()))
|
||||
);
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>('π').parse_peek(b"\xCF\x80"),
|
||||
Ok((Default::default(), "π".as_bytes()))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, Partial<&[u8]>, ErrMode<InputError<_>>>('π')
|
||||
.parse_peek(Partial::new(b"\xCF\x803.14")),
|
||||
Ok((Partial::new(&b"3.14"[..]), "π".as_bytes()))
|
||||
);
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>('π').parse_peek(b"\xCF\x80"),
|
||||
Ok((Default::default(), "π".as_bytes()))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>('π').parse_peek(b"\xCF\x803.14"),
|
||||
Ok((&b"3.14"[..], "π".as_bytes()))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>(AsciiCaseless('a')).parse_peek(b"ABCxyz"),
|
||||
Ok((&b"BCxyz"[..], &b"A"[..]))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>('a').parse_peek(b"ABCxyz"),
|
||||
Err(Backtrack(InputError::at(&b"ABCxyz"[..],)))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>(AsciiCaseless('π')).parse_peek(b"\xCF\x803.14"),
|
||||
Ok((&b"3.14"[..], "π".as_bytes()))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, _, ErrMode<InputError<_>>>(AsciiCaseless('🧑')).parse_peek("🧑你好"),
|
||||
Ok(("你好", "🧑"))
|
||||
);
|
||||
|
||||
let mut buffer = [0; 4];
|
||||
let input = '\u{241b}'.encode_utf8(&mut buffer);
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>(AsciiCaseless('␛'))
|
||||
.parse_peek(input.as_bytes()),
|
||||
Ok((&b""[..], [226, 144, 155].as_slice()))
|
||||
);
|
||||
|
||||
assert_eq!(
|
||||
literal::<_, &[u8], ErrMode<InputError<_>>>('-').parse_peek(b"\xCF\x80"),
|
||||
Err(Backtrack(InputError::at(&b"\xCF\x80"[..],)))
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn tokenslice_location() {
|
||||
#[derive(Clone, Debug)]
|
||||
struct Token {
|
||||
span: core::ops::Range<usize>,
|
||||
}
|
||||
|
||||
impl Location for Token {
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
self.span.end
|
||||
}
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> usize {
|
||||
self.span.start
|
||||
}
|
||||
}
|
||||
|
||||
let input = [
|
||||
Token { span: 1..9 },
|
||||
Token { span: 11..19 },
|
||||
Token { span: 21..29 },
|
||||
];
|
||||
let mut input = TokenSlice::new(&input);
|
||||
assert_eq!(input.previous_token_end(), 1);
|
||||
|
||||
// Parse operation
|
||||
assert_eq!(input.current_token_start(), 1);
|
||||
let _ = input.next_token();
|
||||
assert_eq!(input.previous_token_end(), 9);
|
||||
|
||||
// Parse operation
|
||||
assert_eq!(input.current_token_start(), 11);
|
||||
let _ = input.next_token();
|
||||
assert_eq!(input.previous_token_end(), 19);
|
||||
|
||||
// Parse operation
|
||||
assert_eq!(input.current_token_start(), 21);
|
||||
let _ = input.next_token();
|
||||
assert_eq!(input.previous_token_end(), 29);
|
||||
|
||||
assert_eq!(input.current_token_start(), 29);
|
||||
}
|
||||
+314
@@ -0,0 +1,314 @@
|
||||
use crate::error::Needed;
|
||||
use crate::stream::Checkpoint;
|
||||
use crate::stream::Compare;
|
||||
use crate::stream::CompareResult;
|
||||
use crate::stream::Location;
|
||||
use crate::stream::Offset;
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
use crate::stream::Recover;
|
||||
use crate::stream::SliceLen;
|
||||
use crate::stream::Stream;
|
||||
use crate::stream::StreamIsPartial;
|
||||
use crate::stream::UpdateSlice;
|
||||
use core::iter::Enumerate;
|
||||
use core::slice::Iter;
|
||||
|
||||
/// Specialized input for parsing lexed tokens
|
||||
///
|
||||
/// Helpful impls
|
||||
/// - Any `PartialEq` type (e.g. a `TokenKind` or `&str`) can be used with
|
||||
/// [`literal`][crate::token::literal]
|
||||
/// - A `PartialEq` for `&str` allows for using `&str` as a parser for tokens
|
||||
/// - [`ContainsToken`][crate::stream::ContainsToken] for `T` to for parsing with token sets
|
||||
/// - [`Location`] for `T` to extract spans from tokens
|
||||
///
|
||||
/// See also [Lexing and Parsing][crate::_topic::lexing].
|
||||
#[derive(Copy, Clone, PartialEq, Eq)]
|
||||
pub struct TokenSlice<'t, T> {
|
||||
initial: &'t [T],
|
||||
input: &'t [T],
|
||||
}
|
||||
|
||||
impl<'t, T> TokenSlice<'t, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
/// Make a stream to parse tokens
|
||||
#[inline]
|
||||
pub fn new(input: &'t [T]) -> Self {
|
||||
Self {
|
||||
initial: input,
|
||||
input,
|
||||
}
|
||||
}
|
||||
|
||||
/// Reset the stream to the start
|
||||
///
|
||||
/// This is useful for formats that encode a graph with addresses relative to the start of the
|
||||
/// input.
|
||||
#[doc(alias = "fseek")]
|
||||
#[inline]
|
||||
pub fn reset_to_start(&mut self) {
|
||||
let start = self.initial.checkpoint();
|
||||
self.input.reset(&start);
|
||||
}
|
||||
|
||||
/// Iterate over consumed tokens starting with the last emitted
|
||||
///
|
||||
/// This is intended to help build up appropriate context when reporting errors.
|
||||
#[inline]
|
||||
pub fn previous_tokens(&self) -> impl Iterator<Item = &'t T> {
|
||||
let offset = self.input.offset_from(&self.initial);
|
||||
self.initial[0..offset].iter().rev()
|
||||
}
|
||||
}
|
||||
|
||||
/// Track locations by implementing [`Location`] on the Token.
|
||||
impl<T> TokenSlice<'_, T>
|
||||
where
|
||||
T: Location,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> Option<usize> {
|
||||
let index = self.input.offset_from(&self.initial);
|
||||
index
|
||||
.checked_sub(1)
|
||||
.map(|i| self.initial[i].previous_token_end())
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> Option<usize> {
|
||||
self.input.first().map(|t| t.current_token_start())
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> Default for TokenSlice<'_, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
fn default() -> Self {
|
||||
Self::new(&[])
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> core::ops::Deref for TokenSlice<'_, T> {
|
||||
type Target = [T];
|
||||
|
||||
fn deref(&self) -> &Self::Target {
|
||||
self.input
|
||||
}
|
||||
}
|
||||
|
||||
impl<T: core::fmt::Debug> core::fmt::Debug for TokenSlice<'_, T> {
|
||||
fn fmt(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.fmt(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> SliceLen for TokenSlice<'_, T> {
|
||||
#[inline(always)]
|
||||
fn slice_len(&self) -> usize {
|
||||
self.input.slice_len()
|
||||
}
|
||||
}
|
||||
|
||||
impl<'t, T> Stream for TokenSlice<'t, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
type Token = &'t T;
|
||||
type Slice = &'t [T];
|
||||
|
||||
type IterOffsets = Enumerate<Iter<'t, T>>;
|
||||
|
||||
type Checkpoint = Checkpoint<&'t [T], Self>;
|
||||
|
||||
#[inline(always)]
|
||||
fn iter_offsets(&self) -> Self::IterOffsets {
|
||||
self.input.iter().enumerate()
|
||||
}
|
||||
#[inline(always)]
|
||||
fn eof_offset(&self) -> usize {
|
||||
self.input.eof_offset()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn next_token(&mut self) -> Option<Self::Token> {
|
||||
let (token, next) = self.input.split_first()?;
|
||||
self.input = next;
|
||||
Some(token)
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn peek_token(&self) -> Option<Self::Token> {
|
||||
self.input.first()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn offset_for<P>(&self, predicate: P) -> Option<usize>
|
||||
where
|
||||
P: Fn(Self::Token) -> bool,
|
||||
{
|
||||
self.input.iter().position(predicate)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn offset_at(&self, tokens: usize) -> Result<usize, Needed> {
|
||||
self.input.offset_at(tokens)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn next_slice(&mut self, offset: usize) -> Self::Slice {
|
||||
self.input.next_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn next_slice_unchecked(&mut self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.next_slice_unchecked(offset) }
|
||||
}
|
||||
#[inline(always)]
|
||||
fn peek_slice(&self, offset: usize) -> Self::Slice {
|
||||
self.input.peek_slice(offset)
|
||||
}
|
||||
#[inline(always)]
|
||||
unsafe fn peek_slice_unchecked(&self, offset: usize) -> Self::Slice {
|
||||
// SAFETY: Passing up invariants
|
||||
unsafe { self.input.peek_slice_unchecked(offset) }
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn checkpoint(&self) -> Self::Checkpoint {
|
||||
Checkpoint::<_, Self>::new(self.input)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn reset(&mut self, checkpoint: &Self::Checkpoint) {
|
||||
self.input = checkpoint.inner;
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn raw(&self) -> &dyn core::fmt::Debug {
|
||||
#![allow(deprecated)]
|
||||
self.input.raw()
|
||||
}
|
||||
|
||||
fn trace(&self, f: &mut core::fmt::Formatter<'_>) -> core::fmt::Result {
|
||||
self.input.trace(f)
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> Location for TokenSlice<'_, T>
|
||||
where
|
||||
T: Location,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn previous_token_end(&self) -> usize {
|
||||
self.previous_token_end()
|
||||
.or_else(|| self.current_token_start())
|
||||
.unwrap_or(0)
|
||||
}
|
||||
#[inline(always)]
|
||||
fn current_token_start(&self) -> usize {
|
||||
self.current_token_start()
|
||||
.or_else(|| self.previous_token_end())
|
||||
.unwrap_or(0)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(feature = "unstable-recover")]
|
||||
#[cfg(feature = "std")]
|
||||
impl<T, E> Recover<E> for TokenSlice<'_, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn record_err(
|
||||
&mut self,
|
||||
_token_start: &Self::Checkpoint,
|
||||
_err_start: &Self::Checkpoint,
|
||||
err: E,
|
||||
) -> Result<(), E> {
|
||||
Err(err)
|
||||
}
|
||||
|
||||
/// Report whether the [`Stream`] can save off errors for recovery
|
||||
#[inline(always)]
|
||||
fn is_recovery_supported() -> bool {
|
||||
false
|
||||
}
|
||||
}
|
||||
|
||||
impl<'t, T> StreamIsPartial for TokenSlice<'t, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
type PartialState = <&'t [T] as StreamIsPartial>::PartialState;
|
||||
|
||||
#[inline]
|
||||
fn complete(&mut self) -> Self::PartialState {
|
||||
#![allow(clippy::semicolon_if_nothing_returned)]
|
||||
self.input.complete()
|
||||
}
|
||||
|
||||
#[inline]
|
||||
fn restore_partial(&mut self, state: Self::PartialState) {
|
||||
self.input.restore_partial(state);
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial_supported() -> bool {
|
||||
<&[T] as StreamIsPartial>::is_partial_supported()
|
||||
}
|
||||
|
||||
#[inline(always)]
|
||||
fn is_partial(&self) -> bool {
|
||||
self.input.is_partial()
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> Offset for TokenSlice<'_, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &Self) -> usize {
|
||||
self.offset_from(&other.checkpoint())
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> Offset<<TokenSlice<'_, T> as Stream>::Checkpoint> for TokenSlice<'_, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn offset_from(&self, other: &<TokenSlice<'_, T> as Stream>::Checkpoint) -> usize {
|
||||
self.checkpoint().offset_from(other)
|
||||
}
|
||||
}
|
||||
|
||||
impl<T, O> Compare<O> for TokenSlice<'_, T>
|
||||
where
|
||||
T: PartialEq<O> + Eq,
|
||||
{
|
||||
#[inline]
|
||||
fn compare(&self, t: O) -> CompareResult {
|
||||
if let Some(token) = self.first() {
|
||||
if *token == t {
|
||||
CompareResult::Ok(1)
|
||||
} else {
|
||||
CompareResult::Error
|
||||
}
|
||||
} else {
|
||||
CompareResult::Incomplete
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl<T> UpdateSlice for TokenSlice<'_, T>
|
||||
where
|
||||
T: core::fmt::Debug + Clone,
|
||||
{
|
||||
#[inline(always)]
|
||||
fn update_slice(mut self, inner: Self::Slice) -> Self {
|
||||
self.input = <&[T] as UpdateSlice>::update_slice(self.input, inner);
|
||||
self
|
||||
}
|
||||
}
|
||||
+1094
File diff suppressed because it is too large.
Load diff
+2895
File diff suppressed because it is too large.
Load diff
Reference in new issue
Block a user