Lab 8

Level 1 Classification by Scott Groce
Error Matrix Ground Truth Data
Polygon Class 1 2 3 4 5 6 7 Grand Total
1 273 1 6 18 6 4 4 312
4 29 0 0 9 0 1 9 28
5 14 0 0 0 1 0 0 15
7 1 0 0 0 0 0 0 1
Grand Total 317 1 6 27 7 5 13 376

Accuracy Assessment
Class User's Accuracy Producer's Accuracy
1 87.5% 86.1%
2 0.0% 0.0%
3 0.0% 0.0%
4 18.8% 33.3%
5 6.7% 14.4%
6 0.0% 0.0%
7 0.0% 0.0%
Overall accuracy: 75.3%

Level 1 Classification by Levi Lepping
Error Matrix Ground Truth Data
Polygon Class 1 2 3 4 5 6 7 Grand Total
1 274 1 6 17 6 4 2 310
4 3 0 0 12 1 0 5 21
5 15 0 0 0 1 0 0 16
Grand Total 292 1 6 29 8 4 7 347

Accuracy Assessment
Class User's Accuracy Producer's Accuracy
1 88.4% 93.8%
2 0.0% 0.0%
3 0.0% 0.0%
4 14.3% 41.4%
5 6.3% 12.5%
6 0.0% 0.0%
7 0.0% 0.0%
Overall accuracy: 82.7%

Discussion

Level 1 Classification
First, a brief definition of user's and producer's accuracy so everybody knows what I'm talking about. User's accuracy is the probability that a sample from the classified data actually represents that category on the ground. So if a user of the classified map where to randomly pick a point on the map, this measure communicates how likely it is that the point the user picks is actually of that land class in reality. Producer's accuracy is the probability that all the ground truth points surveyed were classified under the same land use/land cover class by the producer of the map as they were when the points were taken. Overall, Levi's Level 1 classification is 7.4% more accuracte than Scott's. Levi's classification is slightly more accuracte in both user's accuracy and producer's accuracy for the largest class of the classification, Class 1 (Urban or Built-Up Land). Scott's classification is a little more accurate in identifying Class 4 (Forest Land) for users than Levi's while Levi's classification has a higher producer's accuracy for Class 4, but both accuracies for both classifications are still well below 50%. Scott's classification is more accurate in classifying Class 5 (Water) than Levi's, but both measures of accuracy are well below even 25%. Both classifications have 0% accuracy in both producer's and user's accuracy for Class 2 (Agricultural Land), Class 3 (Rangeland), Class 6 (Wetland), and Class 7 (Barren Land). Levi's map is the more accurate map. The field data is of questionable quality. Specific points on the ground when a person is standing there in the field can have quite a different use than when one is looking at the point on an aerial photograph. This can lead to different land use classifications. Also, the aerial photograph was taken in April 2002 and the classifications were completed in February 2003. Many land use changes could have occurred in that time, especially in an urban area like Bellingham. Another point about the field data is that there are far more points taken in Class 1 (Urban and Built-Up) than any other class combined. This can lead to highly accurate maps of Class 1 areas, but accuracies for the other classes are questionable because there are so few points taken in those areas. This can be seen in the sometimes very dramatic differences in user's accuracy and producer's accuracy. User's accuracy can be quite low while at the same time, producer's accuracy can be quite high. This is because of so few points being recorded in those areas. Conversely, accuracies for Class 1 are nearly identical because of the high number of points taken in those areas.

Level 2 Classification by Scott Groce
Error Matrix Ground Truth Data
Polygon Class 10 70 11 12 13 14 15 16 17 21 31 33 41 43 51 52 61 62 76 Grand Total
11 0 0 83 22 0 11 0 0 1 1 3 1 4 9 0 5 0 1 0 141
12 18 4 4 93 0 9 4 13 3 0 2 0 1 4 1 0 0 3 0 159
13 0 0 0 2 3 6 1 0 0 0 0 0 0 0 0 0 0 0 0 12
42 0 0 0 23 0 2 0 1 2 0 0 0 0 8 0 0 1 0 9 46
43 0 0 1 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 2
52 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 3
54 3 0 0 1 0 4 0 0 4 0 0 0 0 0 0 0 0 0 0 12
76 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 1
Grand Total 21 4 88 143 3 32 5 15 10 1 5 1 5 22 1 6 1 4 9 376

Accuracy Assessment
Class User's Accuracy Producer's Accuracy
10 0.0% 0.0%
70 0.0% 0.0%
11 58.9% 94.3%
12 58.5% 65.0%
13 25.0% 100.0%
14 0.0% 0.0%
15 0.0% 0.0%
16 0.0% 0.0%
17 0.0% 0.0%
21 0.0% 0.0%
31 0.0% 0.0%
33 0.0% 0.0%
41 0.0% 0.0%
43 50.0% 4.5%
51 0.0% 0.0%
52 33.3% 16.7%
54 0.0% 0.0%
61 0.0% 0.0%
62 0.0% 0.0%
76 0.0% 0.0%
Overall Accuracy: 48.1%

Level 2 Classification by Levi Lepping
Error Matrix Ground Truth Data
Polygon Class 11 12 13 14 15 16 17 21 31 33 41 43 51 52 62 76 Grand Total
11 82 46 0 23 3 3 5 1 3 1 5 12 1 5 4 2 196
12 3 35 0 0 1 0 0 0 2 0 0 0 0 0 0 0 41
13 0 1 1 0 0 0 0 0 0 0 0 0 0 0 0 0 2
15 0 50 2 3 1 11 1 0 0 0 0 0 0 0 0 0 68
17 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 0 3
42 0 0 0 2 0 0 0 0 0 0 0 4 0 0 0 5 9
43 3 0 0 0 0 0 0 0 0 0 0 8 1 0 0 0 12
52 0 1 0 2 0 0 1 0 0 0 0 0 0 1 0 0 4
54 0 8 0 3 0 0 1 0 0 0 0 0 0 0 0 0 12
Grand Total 88 141 3 31 5 15 9 1 5 1 5 24 2 6 4 7 347

Accuracy Assessment
Class User's Accuracy Producer's Accuracy
11 41.8% 93.2%
12 85.4% 24.8%
13 50.0% 33.3%
14 0.0% 0.0%
15 1.5% 20.0%
16 0.0% 0.0%
17 66.7% 22.2%
21 0.0% 0.0%
31 0.0% 0.0%
33 0.0% 0.0%
41 0.0% 0.0%
42 0.0% 0.0%
43 66.7% 33.3%
51 0.0% 0.0%
52 25.0% 16.7%
54 0.0% 0.0%
62 0.0% 0.0%
76 0.0% 0.0%
Overall Accuracy: 37.5%

Discussion

Level 2 Classification
Overall, Scott's Level 2 classification is 10.6% more accuracte than Levi's. Scott's classification is slightly more accuracte in both user's accuracy and producer's accuracy for the largest class of the classification, Class 11 (Residential). This classification was plagued by many of the same problems as was the level 1 classification for both Scott and Levi. In addition to those previously noted possible inaccuracies, there is another that could have made Levi's overall accuracy lower than Scott's. Scott actually had fewer classes than Levi. This means that there were fewer chances for Scott's classification to be wrong. Neither classification is close to the accepted level of accuracy of 80% and so neither classification can be trusted to provide reliable land use/land class data.