-
Notifications
You must be signed in to change notification settings - Fork 11
Expand file tree
/
Copy pathevaluation_OmniDocBench_markdown_text.txt
More file actions
158 lines (139 loc) · 9.22 KB
/
evaluation_OmniDocBench_markdown_text.txt
File metadata and controls
158 lines (139 loc) · 9.22 KB
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
OmniDocBench size: 981
OmniDocBench markdown_text BLEU: mean=0.25 median=0.02 std=0.35
| BLEU | prob [%] | acc [%] | 1-acc [%] | total |
|----------------|------------|-----------|-------------|---------|
| (0.000, 0.050] | 56.98 | 0 | 100 | 559 |
| (0.050, 0.100] | 3.57 | 56.98 | 43.02 | 35 |
| (0.100, 0.150] | 2.55 | 60.55 | 39.45 | 25 |
| (0.150, 0.200] | 2.96 | 63.1 | 36.9 | 29 |
| (0.200, 0.250] | 2.75 | 66.06 | 33.94 | 27 |
| (0.250, 0.300] | 2.24 | 68.81 | 31.19 | 22 |
| (0.300, 0.350] | 0.82 | 71.05 | 28.95 | 8 |
| (0.350, 0.400] | 0.82 | 71.87 | 28.13 | 8 |
| (0.400, 0.450] | 0.71 | 72.68 | 27.32 | 7 |
| (0.450, 0.500] | 1.22 | 73.39 | 26.61 | 12 |
| (0.500, 0.550] | 0.61 | 74.62 | 25.38 | 6 |
| (0.550, 0.600] | 0.92 | 75.23 | 24.77 | 9 |
| (0.600, 0.650] | 1.22 | 76.15 | 23.85 | 12 |
| (0.650, 0.700] | 2.34 | 77.37 | 22.63 | 23 |
| (0.700, 0.750] | 1.94 | 79.71 | 20.29 | 19 |
| (0.750, 0.800] | 3.47 | 81.65 | 18.35 | 34 |
| (0.800, 0.850] | 4.38 | 85.12 | 14.88 | 43 |
| (0.850, 0.900] | 4.18 | 89.5 | 10.5 | 41 |
| (0.900, 0.950] | 4.59 | 93.68 | 6.32 | 45 |
| (0.950, 1.000] | 1.73 | 98.27 | 1.73 | 17 |
OmniDocBench markdown_text F1: mean=0.44 median=0.33 std=0.36
| F1 | prob [%] | acc [%] | 1-acc [%] | total |
|----------------|------------|-----------|-------------|---------|
| (0.000, 0.050] | 14.27 | 0 | 100 | 140 |
| (0.050, 0.100] | 9.79 | 14.27 | 85.73 | 96 |
| (0.100, 0.150] | 7.75 | 24.06 | 75.94 | 76 |
| (0.150, 0.200] | 4.08 | 31.8 | 68.2 | 40 |
| (0.200, 0.250] | 4.89 | 35.88 | 64.12 | 48 |
| (0.250, 0.300] | 7.03 | 40.77 | 59.23 | 69 |
| (0.300, 0.350] | 4.49 | 47.81 | 52.19 | 44 |
| (0.350, 0.400] | 3.36 | 52.29 | 47.71 | 33 |
| (0.400, 0.450] | 4.38 | 55.66 | 44.34 | 43 |
| (0.450, 0.500] | 1.22 | 60.04 | 39.96 | 12 |
| (0.500, 0.550] | 0.92 | 61.26 | 38.74 | 9 |
| (0.550, 0.600] | 0.92 | 62.18 | 37.82 | 9 |
| (0.600, 0.650] | 0.61 | 63.1 | 36.9 | 6 |
| (0.650, 0.700] | 1.22 | 63.71 | 36.29 | 12 |
| (0.700, 0.750] | 1.83 | 64.93 | 35.07 | 18 |
| (0.750, 0.800] | 1.02 | 66.77 | 33.23 | 10 |
| (0.800, 0.850] | 3.36 | 67.79 | 32.21 | 33 |
| (0.850, 0.900] | 12.03 | 71.15 | 28.85 | 118 |
| (0.900, 0.950] | 11.42 | 83.18 | 16.82 | 112 |
| (0.950, 1.000] | 5.4 | 94.6 | 5.4 | 53 |
OmniDocBench markdown_text precision: mean=0.50 median=0.50 std=0.35
| precision | prob [%] | acc [%] | 1-acc [%] | total |
|----------------|------------|-----------|-------------|---------|
| (0.000, 0.050] | 11.01 | 0 | 100 | 108 |
| (0.050, 0.100] | 6.42 | 11.01 | 88.99 | 63 |
| (0.100, 0.150] | 6.01 | 17.43 | 82.57 | 59 |
| (0.150, 0.200] | 4.18 | 23.45 | 76.55 | 41 |
| (0.200, 0.250] | 4.99 | 27.62 | 72.38 | 49 |
| (0.250, 0.300] | 6.32 | 32.62 | 67.38 | 62 |
| (0.300, 0.350] | 3.87 | 38.94 | 61.06 | 38 |
| (0.350, 0.400] | 2.14 | 42.81 | 57.19 | 21 |
| (0.400, 0.450] | 3.16 | 44.95 | 55.05 | 31 |
| (0.450, 0.500] | 1.63 | 48.11 | 51.89 | 16 |
| (0.500, 0.550] | 7.14 | 49.75 | 50.25 | 70 |
| (0.550, 0.600] | 1.83 | 56.88 | 43.12 | 18 |
| (0.600, 0.650] | 0.82 | 58.72 | 41.28 | 8 |
| (0.650, 0.700] | 2.75 | 59.53 | 40.47 | 27 |
| (0.700, 0.750] | 0.61 | 62.28 | 37.72 | 6 |
| (0.750, 0.800] | 2.14 | 62.9 | 37.1 | 21 |
| (0.800, 0.850] | 3.16 | 65.04 | 34.96 | 31 |
| (0.850, 0.900] | 4.79 | 68.2 | 31.8 | 47 |
| (0.900, 0.950] | 16.72 | 72.99 | 27.01 | 164 |
| (0.950, 1.000] | 10.3 | 89.7 | 10.3 | 101 |
OmniDocBench markdown_text recall: mean=0.47 median=0.37 std=0.34
| recall | prob [%] | acc [%] | 1-acc [%] | total |
|----------------|------------|-----------|-------------|---------|
| (0.000, 0.050] | 14.17 | 0 | 100 | 139 |
| (0.050, 0.100] | 2.34 | 14.17 | 85.83 | 23 |
| (0.100, 0.150] | 3.87 | 16.51 | 83.49 | 38 |
| (0.150, 0.200] | 6.42 | 20.39 | 79.61 | 63 |
| (0.200, 0.250] | 6.63 | 26.81 | 73.19 | 65 |
| (0.250, 0.300] | 8.66 | 33.44 | 66.56 | 85 |
| (0.300, 0.350] | 6.42 | 42.1 | 57.9 | 63 |
| (0.350, 0.400] | 3.67 | 48.52 | 51.48 | 36 |
| (0.400, 0.450] | 3.98 | 52.19 | 47.81 | 39 |
| (0.450, 0.500] | 1.43 | 56.17 | 43.83 | 14 |
| (0.500, 0.550] | 2.55 | 57.59 | 42.41 | 25 |
| (0.550, 0.600] | 1.43 | 60.14 | 39.86 | 14 |
| (0.600, 0.650] | 1.22 | 61.57 | 38.43 | 12 |
| (0.650, 0.700] | 1.22 | 62.79 | 37.21 | 12 |
| (0.700, 0.750] | 1.12 | 64.02 | 35.98 | 11 |
| (0.750, 0.800] | 2.55 | 65.14 | 34.86 | 25 |
| (0.800, 0.850] | 3.98 | 67.69 | 32.31 | 39 |
| (0.850, 0.900] | 13.35 | 71.66 | 28.34 | 131 |
| (0.900, 0.950] | 9.07 | 85.02 | 14.98 | 89 |
| (0.950, 1.000] | 5.91 | 94.09 | 5.91 | 58 |
OmniDocBench markdown_text edit_distance: mean=0.67 median=0.84 std=0.34
| edit_distance | prob [%] | acc [%] | 1-acc [%] | total |
|-----------------|------------|-----------|-------------|---------|
| (0.000, 0.050] | 1.12 | 0 | 100 | 11 |
| (0.050, 0.100] | 3.57 | 1.12 | 98.88 | 35 |
| (0.100, 0.150] | 10.09 | 4.69 | 95.31 | 99 |
| (0.150, 0.200] | 3.36 | 14.78 | 85.22 | 33 |
| (0.200, 0.250] | 4.49 | 18.14 | 81.86 | 44 |
| (0.250, 0.300] | 2.34 | 22.63 | 77.37 | 23 |
| (0.300, 0.350] | 2.65 | 24.97 | 75.03 | 26 |
| (0.350, 0.400] | 1.53 | 27.62 | 72.38 | 15 |
| (0.400, 0.450] | 1.73 | 29.15 | 70.85 | 17 |
| (0.450, 0.500] | 1.02 | 30.89 | 69.11 | 10 |
| (0.500, 0.550] | 2.04 | 31.91 | 68.09 | 20 |
| (0.550, 0.600] | 1.22 | 33.94 | 66.06 | 12 |
| (0.600, 0.650] | 1.53 | 35.17 | 64.83 | 15 |
| (0.650, 0.700] | 1.33 | 36.7 | 63.3 | 13 |
| (0.700, 0.750] | 1.73 | 38.02 | 61.98 | 17 |
| (0.750, 0.800] | 5.4 | 39.76 | 60.24 | 53 |
| (0.800, 0.850] | 6.01 | 45.16 | 54.84 | 59 |
| (0.850, 0.900] | 8.36 | 51.17 | 48.83 | 82 |
| (0.900, 0.950] | 13.46 | 59.53 | 40.47 | 132 |
| (0.950, 1.000] | 27.01 | 72.99 | 27.01 | 265 |
OmniDocBench markdown_text meteor: mean=0.41 median=0.28 std=0.35
| meteor | prob [%] | acc [%] | 1-acc [%] | total |
|----------------|------------|-----------|-------------|---------|
| (0.000, 0.050] | 17.53 | 0 | 100 | 172 |
| (0.050, 0.100] | 5.3 | 17.53 | 82.47 | 52 |
| (0.100, 0.150] | 10.5 | 22.83 | 77.17 | 103 |
| (0.150, 0.200] | 7.65 | 33.33 | 66.67 | 75 |
| (0.200, 0.250] | 6.32 | 40.98 | 59.02 | 62 |
| (0.250, 0.300] | 3.87 | 47.3 | 52.7 | 38 |
| (0.300, 0.350] | 3.36 | 51.17 | 48.83 | 33 |
| (0.350, 0.400] | 2.85 | 54.54 | 45.46 | 28 |
| (0.400, 0.450] | 2.24 | 57.39 | 42.61 | 22 |
| (0.450, 0.500] | 2.04 | 59.63 | 40.37 | 20 |
| (0.500, 0.550] | 1.94 | 61.67 | 38.33 | 19 |
| (0.550, 0.600] | 1.33 | 63.61 | 36.39 | 13 |
| (0.600, 0.650] | 2.14 | 64.93 | 35.07 | 21 |
| (0.650, 0.700] | 1.53 | 67.07 | 32.93 | 15 |
| (0.700, 0.750] | 2.45 | 68.6 | 31.4 | 24 |
| (0.750, 0.800] | 3.47 | 71.05 | 28.95 | 34 |
| (0.800, 0.850] | 3.36 | 74.52 | 25.48 | 33 |
| (0.850, 0.900] | 6.52 | 77.88 | 22.12 | 64 |
| (0.900, 0.950] | 11.52 | 84.4 | 15.6 | 113 |
| (0.950, 1.000] | 4.08 | 95.92 | 4.08 | 40 |