We had our early December meeting at the nearby Challenger Elementary School, held there specifically to showcase the ARCH ( A.R.C.H. (Academic enrichment, Recreation and the arts,Community and family services, Healthy Relationships) Programs. These are free to qualified public school students and made possible through a 21st Century Community Learning Center Grant and sponsored by the Michigan Department of Education. Our program is unique because it’s one of a few in which a city, Kentwood (basically Parks Department), partners with our public schools to make this supplemental learning program available to the 15 schools in our city limits. ARCH is funded ultimately by our federal government. It is an attempt to help the bottom 30% of the students (mostly minority from the presentation and observation.) The kids stay after school for up to 3 hours, get a free dinner and snacks, participate in sports and get supplemental teaching in various academic topics, of which math seems to be the main deficit requiring attention.
We in the commission have to approve the funding and resource allocation of this 2.5 million dollar a year program which is currently in its second of a five year cycle. ARCH in Kentwood first started about seven years ago as smaller program and was expanded last year to involve all 15 of the public schools in our city.
It was important to the folks involved to make city commissioner buy into ARCH and so they mounted a two pronged assault. The first was for us to meet at a school and walk through the building observing this program operate so that we could experience the process. You know, the Montessori touch, participate, “play with it with your hands method” of proselytism, if you know what that means,
The second was more conventional didactics; teachers and a few students were present and spoke. The main speech was from yet another coordinator of the program who gave her statistical analysis of how effective the program has been in bringing poorly achieving, minority and often non English speaking students up to grade level. This attempt to create legitimacy by doing an analysis, purportedly a “scientific study” caught my attention. A rough summary suffices for my purposes.
Some 1600 kids in our Kentwood PS (charters, parochial schools need not apply) students, nominated by teachers because they were in the lowest 30% in their classes, are invited to join. About 68% sign on; 500 don’t bother to enroll. Actual participation is about 600 per day, which I calculate to be about 37% of those eligible. The further confounding issue is that the kids in ARCH are in unstable families, they move frequently to other schools or cities, and have problems with English. There is a huge turnover of kids in the programs. The numbers for older kids were too small for reliable numbers and no one mentioned the high school dropout rate for Kentwood schools.
A lady named Maria had crunched the numbers. She compared the grades of kids in the program with kids who were too successful to merit inclusion with the poorly performing cohort. The original grades differences were fairly marked before ARCH and improved, although not nearly to equality. The implication was that ARCH was worthwhile and might be a harbinger of continued academic achievement among the poorest performers.
No one, including your reporter, openly challenged these unwarranted assertions. There was a general air of satisfaction in the room; we in government had done something good. It’s hard to stand up and explain why and how this assertion was not warranted based on the “scientific analysis.” I did privately share some of my doubts with Maria, but, in public, I’m a coward. It’s a familial trait that I’ll never overcome, so a week later I blog to expiate my guilt.
My main concern here is not whether or not ARCH is good for these kids and worth the 4000 dollar per year per kid cost; my complaint is that 30 or 40 college graduates in the room were happy to spend other people’s money based on an essentially worthless and potentially misleading analysis. The overall arc of reasoning in this and similar studies is; we have an idea that might work, namely prolong the school day to deliver more of the quantity called “education” which will improve the kids. We then have to try it out. After a while, we’ll need to support the original hypothesis by analysing whatever data we can cobble together, and when we find that the data can’t support the effort, we assert the legitimacy of the original hypothesis anyway and spend more money on it..
As with almost all social sciences research, is that these “studies” have become thinly disguised marketing studies, in this case, an attempt to convince the city commission to support this program as it is now structured. The criticisms that I as a somewhat trained researcher (12 articles in the juried medical literature) advanced have to do with selection biases. Why did some parents decide to send their kids to ARCH, and how do these differ? Why compare them to successful kids? Wouldn’t the proper comparison group be those who were eligible for the program but whose parents didn’t let their kids participate? Or, maybe compare to those kids whose parents who signed them into the program but who didn’t bother to show up?
Beyond the selection biases lie statistical problems of treating changing populations of kids as they move in and out of the district as though they had all benefited from the entire program. Maybe the reported improvements were caused when smarter kids moved in, or maybe not. The unanswered question of drop outs in the senior high school levels was troubling; what are the dropout rates in Kentwood public schools? We had no way of knowing.
There are more abstract considerations like why ARCH is not offered at charter or parochial schools in Kentwood. What effect does keeping these kids away from their families for 11 hours each day have on their emotional involvement in their own families?
The physical sciences concern inanimate objects and delve into reproducible, boring and non human phenomena. They can be used to make things like rockets, computers and allow wealth creation to pay for stuff like ARCH.
The social sciences attempt to explore the most fascinating of all subjects, man himself. The social sciences try to imitate the physical sciences by marshalling man’s actions as though they were cause and effect instead of being motivated. The subject never changes his nature but constantly changes his attentions and intentions; the interplay is too complex and shifting to be predictable but is endlessly thought provoking. The social sciences produce studies riddled with non valid and not reliable results.They are useless in predicting much of anything and at best deal in hopes and fantasies. The social sciences can entertain us with analyses of why something, say, the Great Depression, the rise of Islamism or slavery happened in the past. None of these analyses agree and these differences provoke argument, foster major industries in the conspiracy theories, government programs and of similar charlatans. They make money from gullible students who major in these topics and from doing marketing studies that influence the news media, government officials and an inattentive public.
In other words, the social sciences do not produce evidence that can be used for human engineering, but only for human entertainment. I suppose that most of the 30 or 40 college grads that evening at Challenger School left with a warm fuzzy feeling and a few of us fantasized that it was of the wool being pulled over our eyes.