Increasing feedback for EIA practitioners

In October 2012, science took a hard hit in Italy when 7 scientists were convicted for manslaughter for failing to predict a 2009 6.3 magnitude earthquake. The judge ruled that the defendants gave a false sense of security to the population of Aquila, Italy, when downplaying the importance of tremors hitting the region of Abruzzo. The earthquake was devastating, taking the lives of over 300 people.

News Report on the Aquila eathquake convictions

As a geologist, the thought of being held accountable for failing to predict the unpredictable is terrifying and I want to make it clear that I in no way defend or agree with the judge’s ruling.

However, as an EIA practitioner to be, I believe that an essential part of this profession is to evaluate the accuracy and correctness of past EIA’s predictions. Monitoring and follow-up is considered to follow an EIA and are usually viewed as tools to ensure proper implementation of mitigation measures as well as potential indicators for unexpected impacts. However, its role in trial and error is often secondary or even forgotten. I believe that is a fundamental failure of the current EIA system. In order to not repeat the same omissions and mistakes from one EIA to the next, practitioners should have to review past EIA’s based on the findings of post-monitoring. This would not be for evaluation purposes of the employee but to understand where in the process something might have been overlooked and how to improve on this in the future.

A lack of feedback is a critical impediment to learning and tremendously decreases the value of experience. This is evident in the medical field. A 2003 study by Conant and Sickles found that the diagnostic accuracy of mammographers does not improve over time, as would be expected with increasing experience. This is due to the lack of feedback, since most mammographers are not part of the medical team which then treats the patient and are usually unaware of a misdiagnosis. Furthermore, even when informed, the time lag between the diagnostic and the discovery of a mistake is large enough that details of the cases are long forgotten. On the other hand, surgeons do become better with experience, as the feedback is immediate (Flum & al., 2002 ; Sosa & al., 1998). Which procedures works and does not work in the surgery room is apparent to the surgeon who can then adapt and improve based on responses.

Of course, EIA appears to be more in the camp of the mammographers than surgeons in the sense that post-monitoring is often conducted by a different team than the one which conducted the EIA. Furthermore, feedback is spaced out in times in that failures to predict impacts will often only be discovered years in the future.  I believe that implementing a few procedures can go a long way in providing practitioners feedback. For example, firms could require a yearly report should be written, comparing the predictions to the results for the initial years following the EIA. Since time is obviously a restriction, a brief but to the point review would be sufficient to ensure that the practitioner is aware of the post-monitoring findings. Another possibility would be to instill a process similar to that of peer-review for scientific papers. Again, time and resources are constraints, but having a different EIA team read over a submitted EIA report would provide the necessary feedback. This review group would be able to point out the limits and the flaws to the current report and simulate the immediate feedback a surgeon receives. Furthermore, this would help share experience acquired by the various professionals.

Finally, it is worth remembering that not all feedback is good feedback:

Calvin_Hobbes_Data_Quality

References :

C.A. Beam, E.F.Conant, and E.A. Sickles (2003), “Association of Volume and Volume-Independent Factors with Accuracy in Screening Mammogram Interpretation,” Journal of  the National Cancer Institute 95, 282-90.

Sosa JA, Bowman HM, Tielsch JM, Powe NR, Gordon TA, Udelsman R (1998), “ The importance of surgeon experience for clinical and economic outcomes from thyroidectomy, ”  Annals of Surgery, 228:320–330.

Flum DR, Koepsell T, Heagerty P, Pellegrini CA (2002), “The nationwide frequency of major adverse outcomes in antireflux surgeryand the role of surgeon experience, 1992–1997, ” Journal of the American College of Surgeons, 195:611–618.

Hooper, John. “Italian Scientists Convicted for ‘false Assurances’ before Earthquake.” The Guardian. Published online on October, 22nd 2012. Last accessed on January 15th, 2013. Available at <http://www.guardian.co.uk/world/2012/oct/22/italian-scientists-jailed-earthquake-aquila&gt;.

Calvin and Hobbes Comic Strip. Copyright Bill Watterson.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s