IndexOutOfRangeException serializing large strings #103155
Description
Description
Json serializer could throw IndexOutOfRangeException exception on large strings serialization. Looks like string should be in 100Mb-160Mb to fail. Larger strings fail with expected string too big exception.
Reproduction Steps
using System.Text;
using System.Text.Json;
internal class Program
{
static void Main(string[] args)
{
var rnd = new Random();
while (true)
{
var length = rnd.Next(100_000_000, 160_000_000);
var sb = new StringBuilder();
while (sb.Length < length)
{
var allowSurrogate = length - sb.Length > 1;
_ = sb.Append(Char(rnd));
}
try
{
_ = JsonSerializer.Serialize(sb.ToString(), JsonSerializerOptions.Default);
}
catch (ArgumentException ex) when (ex.Message.Contains("too large and not supported"))
{
// ignore expected errors
Console.WriteLine($"Too big: {ex.Message}");
}
//catch (IndexOutOfRangeException)
//{
// Console.WriteLine($"Crashed!!! String size: {sb.Length}");
//}
}
static string Char(Random rnd)
{
if (rnd.Next(0, 10) == 0)
{
while (true)
{
var chr = ((char)(ushort)rnd.Next(0xd800, 0xdbff)).ToString()
+ ((char)(ushort)rnd.Next(0xdc00, 0xdfff)).ToString();
if (char.IsSurrogatePair(chr, 0))
{
return chr;
}
}
}
while (true)
{
var chr = (char)(ushort)rnd.Next(ushort.MaxValue);
if (chr is < (char)0xD800 or > (char)0xDFFF)
{
return chr.ToString();
}
}
}
}
}Expected behavior
No exceptions if string size in allowed limits and ArgumentException when limit reached (as it happens now for strings > ~160Mb).
Actual behavior
Unhandled exception. System.IndexOutOfRangeException: Index was outside the bounds of the array.
at System.Text.Json.Utf8JsonWriter.WriteStringMinimized(ReadOnlySpan`1 escapedValue)
at System.Text.Json.Utf8JsonWriter.WriteStringEscapeValue(ReadOnlySpan`1 value, Int32 firstEscapeIndexVal)
at System.Text.Json.Utf8JsonWriter.WriteStringEscape(ReadOnlySpan`1 value)
at System.Text.Json.Utf8JsonWriter.WriteStringValue(ReadOnlySpan`1 value)
at System.Text.Json.Serialization.Converters.StringConverter.Write(Utf8JsonWriter writer, String value, JsonSerializerOptions options)
at System.Text.Json.Serialization.JsonConverter`1.TryWrite(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
at System.Text.Json.Serialization.JsonConverter`1.WriteCore(Utf8JsonWriter writer, T& value, JsonSerializerOptions options, WriteStack& state)
at System.Text.Json.Serialization.Metadata.JsonTypeInfo`1.Serialize(Utf8JsonWriter writer, T& rootValue, Object rootValueBoxed)
at System.Text.Json.JsonSerializer.WriteString[TValue](TValue& value, JsonTypeInfo`1 jsonTypeInfo)
at Program.Main(String[] args)
Regression?
No response
Known Workarounds
As we actually don't need such large strings serialized, we are using custom string converter which trims them to sane length.
Configuration
Runtime: 8.0.5
Arch: Windows x64
Other information
No response