Implement nom-based ESI parser with streaming support #43
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ut esi:vars and recognize basic interpolation.
…little cleanup.
This merge brings in the new nom-based parser implementation up to commit f216023: Key changes: - Added nom dependency for modern parsing capabilities - Introduced new_parse.rs with comprehensive nom-based ESI parser - Added parser_types.rs with new Expr and Operator types for better type safety - Enhanced interpolation handling with process_interpolated_chars function - Improved error handling and debugging output - Fixed HTML tag closure issues in examples - Added support for expression comparisons (matches, matches_i operators) - Better integration between parser and interpreter with consistent types The new parser provides better performance, more robust parsing, and improved maintainability while maintaining full compatibility with existing ESI functionality.
… expressions and nested structures
…pdate related parsing logic
…a nested vector structure for improved handling of events
…nt handling - Introduced a request field in the Fragment struct to retain the original request. - Updated the Processor to utilize the provided fragment response processor. - Added tests to verify behavior with is_escaped_content configuration and response processing.
- Modify the `lower` function to return `Null` for `Value::Null` arguments. - Refactor `process_nom_chunk_to_output` to handle expression evaluation more robustly, including skipping output for `Null` values. - Replace the simple expression evaluator with a more comprehensive parser that supports additional operators and expressions. - Introduce parsing for interpolated strings and standalone ESI expressions. - Add support for logical, comparison, and negation operators in the parser. - Update `parser_types` to include new expression types and operators. - Enhance tests for expression parsing to cover new functionality.
…cess_chunk_to_output
- Introduced `WhenBranch` struct to represent branches in choose blocks. - Updated `Tag` enum to use `WhenBranch` and changed `Chunk` to `Element`. - Modified `Assign` tag to accept `Expr` instead of `String` for value. - Enhanced parsing logic to support interpolated expressions in assignments. - Added tests for long form assignments with interpolation and multiple variables. - Updated existing tests to reflect changes in parser structure and expression handling.
…`*chunk*` to `*element*`
…ess_include function for improved clarity and error management; update parser types to use string slices.
…and responses - Added a new `Processor` struct to handle ESI directives in HTML/XML content. - Implemented methods for processing ESI documents, including handling includes, conditionals, and variable substitutions. - Introduced `PendingFragmentContent` to manage the state of fragment requests. - Created a queue system for processing elements in order, allowing for non-blocking operations. - Added support for try/except blocks in ESI processing. - Implemented a default fragment request dispatcher that uses the request's hostname. - Enhanced error handling and logging throughout the processing flow. - Included detailed documentation and examples for the new functionality.
…spatch for attempts and streamline processing with recursive methods for better clarity and maintainability.
…equest handling into a single method for improved clarity and maintainability.
…handling for better stability and clarity.
- Implement tests for basic ESI include tags, including attributes like src, alt, and onerror. - Add tests for ESI try/except structures, ensuring nested tries are parsed correctly. - Introduce tests for ESI assign tags, covering various value types including strings, integers, variables, and function calls. - Validate parsing of ESI vars in both short and long forms. - Test ESI choose/when/otherwise structures, ensuring multiple when branches are handled correctly. - Ensure content within esi:remove and esi:comment tags is not included in the parsed output. - Verify that esi:text tags preserve their content as plain text, including nested ESI tags. - Add a mixed content test to ensure proper parsing of HTML, expressions, ESI tags, and text.
…input handling - Implement tests to validate the behavior of the streaming parser with incomplete tags, ensuring it returns `Incomplete` when more data is needed. - Validate that the `delimited()` combinator correctly propagates errors and incomplete states from inner parsers. - Test the distinction between `parse()` and `parse_complete()` on incomplete input, confirming that `parse_complete()` treats it as EOF. - Ensure that the parser handles various edge cases, including incomplete attributes, nested tags, and whitespace scenarios. - Document the streaming parser architecture and best practices for handling incomplete input in real-world applications.
…data and improve error propagation in streaming scenarios
- Updated all parser tests in `parser.rs` and `streaming_behavior.rs` to utilize `Bytes` for input, enhancing performance by avoiding unnecessary copies. - Introduced a new test file `zero_copy.rs` to specifically test zero-copy parsing behavior for ESI tags like `esi:try`, `esi:choose`, and `esi:remove`. - Ensured that all tests validate the expected behavior of the parser with complete and incomplete inputs, while maintaining the integrity of the parsed elements.
…avior tests - Updated tests in streaming_behavior.rs to ensure they align with the new parsing logic and behavior. - Cleaned up whitespace in the tests for better readability. - Ensured that the tests validate the complete input parsing behavior correctly.
… zero-copy parsing
…proving performance and memory efficiency
…ml_encode and Processor methods
…ept, esi_attempt, and esi_otherwise functions
… for improved clarity and maintainability
…er and improve parsing logic for HTML tags
…r improved handling of quotes and nested ESI tags
…g with zero-copy parsing
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Introduce a new parser utilizing the nom library for enhanced performance and maintainability. This implementation includes comprehensive support for ESI tags, improved error handling, and streaming processing capabilities. Refactor existing code to integrate the new parser while ensuring compatibility with current functionality. Extensive tests validate the new parser's behavior and performance.