JAR e um artigo sob suspeita

Um periódico como o Journal of Accounting Research - um dos melhores da nossa área - publicou um artigo que está sendo objeto de contestação. O artigo lida com fraude na contabilidade, um assunto relevante. Usando machine learning os autores conseguiram aumentar a taxa de detecção em relação a regressão logística. O problema é que um texto no EcoWatchJournal descobriu falhas no processo. Eis o resumo:

This critique examines the results of an article that applies machine learning to the detection of accounting fraud, published in Journal of Accounting Research. Their key finding is that machine learning improved fraud detection by 70 percent above a previously published logistic regression. The authors make their data and Matlab code available at Github. Using their files, I replicate their study. Upon closer inspection, we see that some fraudulent firms were contained in both the training and test samples, which improves the results of their model, but contradicts what was described in the published paper. I asked the authors about this issue and gratefully received a response. The response is quoted in the present critique. Getting a proper assessment of the potential of machine learning is important, as such techniques and models are relied upon by industry practitioners and regulators, including the Securities and Exchange Commission.

Veja que o problema é sério. Os autores usaram as mesmas empresas na amostra de teste e na amostra de treinamento. Isto é um erro, já que melhora os resultados. 

Em agosto de 2022 os autores publicaram uma errata do artigo. Mas a crítica persistiu no próprio Econ Watch Journal:

This paper treats an erratum published in Journal of Accounting Research (JAR) in August 2022. The erratum was prompted by two critical comments authored by me and published in Econ Journal Watch. The erratum mischaracterizes its authors’ previous research related to the preferred test sample period. More importantly, the authors say that I identified an error in their program code. This is false. Rather, I identified a misidentification within the dataset, a misidentification that was disclosed neither in their original JAR article, nor in the program code appended to that article, nor in their Econ Journal Watch reply to my first comment. Finally, the erratum never addresses why the misidentification occurred, nor why they did not acknowledge the misidentification on the two prior opportunities to do so. I have asked for an investigation at the Journal of Accounting Research into academic research misconduct.

A errata tenta confundir a crítica feita, já que não foi um problema no código, mas o uso da base de dados. Veja que no final, Walker, que levantou a polêmica sugere que é um caso para investigação de má conduta científica. 

via aqui