Users should be able to set certain limits when making network requests or reading from files, such as:
• Initial size: The starting size of data to be fetched or read.
• Read increment size: The size of data to be read next, until the max limit is reached, the data source is empty, or a specific tag is found.
• Max size: The maximum total data to be fetched or read.
This feature will enable partial data fetching, allowing, for example, the parsing of just the header of a web page without needing to fully download or read the entire page.
The functionality should be integrated into the SourceReader component, so it can work seamlessly with both file systems and network URLs.
Users should be able to set certain limits when making network requests or reading from files, such as:
This feature will enable partial data fetching, allowing, for example, the parsing of just the header of a web page without needing to fully download or read the entire page.
The functionality should be integrated into the SourceReader component, so it can work seamlessly with both file systems and network URLs.