101 - Prophylaxis and treatment of whooping cough with a pertussis antigen

This was an unusual study on a vaccine-like method of protecting children from whooping cough. Instead of using whole killed bacterial cells, they grew the cells and filtered them out, leaving only soluble factors the bacteria had produced externally, which they called "antigen." Then they inactivated this antigen so it wouldn't cause any problems but would still induce an immune response, treating it with formaldehyde and mild heat. This was sorta like how tetanus and diphtheria toxoid vaccines were produced.

Then they tested this antigen for how well it could a) treat whooping cough cases, b) prevent catching the disease when given when not at risk, and c) prevent catching the disease when given after being exposed.

Antigen Therapy
They had two main groups: children with whooping cough treated with antigen, and control children with whooping cough. The controls were treated with either vaccine or vaccine + antigen. They don't specify what they mean by "vaccine;" details are pretty sparse in general.

The average time before each group was similar, about 1.5 weeks, but the duration of disease after treatment was about 1.6 weeks for the antigen-treated and 3 weeks for controls, so a pretty big difference.

The number of complications in the groups was different too: About 0.7% of the antigen-treated had complications from the disease (bronchitis, conjunctival hemorrhage, otitis media), while 18% of controls had complications, including some deaths

Antigen Prophylaxis
Of those immunized without first being exposed, about 81% of those who did become exposed didn't get the disease, so they said it was 81% effective. That's not exactly how such trials should work; there should be controls.

They claimed this immunity lasted at least 5 months and up to 2.5 years, but I'm not sure how they determined this.

The same was true of those treated after being exposed: 76% didn't get the disease, but again there were no controls reported.

In terms of adverse reactions, some few patients had a short, mild fever, arm redness, or hives, but nothing worse.

Conclusions
So considering the lack of details, controls, and blinding, and how this approach didn't seem to be considered worthwhile either before or after this study, I'm not convinced it's very valuable.

Reference:

Joslin, C. L. & Christensen, T. A. Prophylaxis and treatment of whooping cough with a pertussis antigen: Report of results. Am J Dis Child 60, 1269–1276 (1940).

099 - Secondary Familial Attack Rates from Pertussis in Vaccinated and Unvaccinated Children

This study is a follow-up analysis of previous results by Kendrick and Eldering of a clinical trial in Grand Rapids, Michigan, of a whole-cell pertussis vaccine.

Rather than the whole study population, this study focused only on 165 families that were definitely exposed to pertussis from one of their own members. The other members' immunity status was known from history of vaccination or the disease itself.

What they saw was that, of all 78 vaccinated subjects, only 28 got whooping cough (36%). All these were under age 7. In contrast, of those without history of vaccination or disease, 79% got sick. Of those unvaccinated under age 7, 92% were attacked (the rate decreased with increasing age of subject). 36% vs. 92% for those most vulnerable? Not bad.

Most of the primary cases, that brought the disease to a family, were in the older category, confirming that older children often bring it to their younger siblings. Interestingly, with the subjects who had the disease before, 5 of the 6 that got sick were over 7 years old, so it seems there's a period after which the immunity is not so great, even after natural infection. The study was set up to analyze that in depth though.

Also noteworthy is that, of 172 primary cases that brought the disease to the families, 157 were unvaccinated (91%), 12 were vaccinated (7%), and 3 had the disease previously. There's some herd immunity in action, perhaps, but it's hard to tell.

This wasn't a very rigorous study (lack of blinding or placebo), but considering that an intimate exposure, such as in the same household, is probably the most difficult to have immunity against, there seems to be some effect. And apparently whooping cough is so contagious that 80-90% get sick when exposed to it this way, if not immune somehow.

References:

Kendrick, P. L. Secondary Familial Attack Rates from Pertussis in Vaccinated and Unvaccinated Children. Am. J. Epidemiol. 32-SectionA, 89–91 (1940).

097 - An Experiment in Immunization Against Influenza with a Formaldehyde-Inactivated Virus

There had been some good results in animals with vaccines against influenza, and possibly some good trials in humans too, though not in others; influenza is tricky though, due to antigenic drift.

So this was another study, in Hungary during an epidemic in 1937. They used formaldehyde-inactivated virus taken from infected mouse lungs, and vaccinated 306 nurses and children in Budapest institutions with a single dose. 336 were controls.

They tested some subjects before and after the vaccination, and found that antibody levels rose a decent amount after the vaccination, 24x on average. One subject got tested before, after the vaccine, and then again after getting the flu, and the disease hadn't increased the levels any higher than the vaccine had (though obviously it wasn't a protective level somehow).

The epidemic was pretty small, so there were only 34 cases total in the study: 20 in the unvaccinated, and 14 in the vaccinated. This wasn't a significant difference, so it was negative.

But they isolated virus from some of the cases and found that it was a different serotype, so the vaccine might not've been good at targeting it anyway. Oh well.

Reference:

Taylor, R. M. & Dreguss, M. An Experiment in Immunization Against Influenza with a Formaldehyde-Inactivated Virus. Am. J. Epidemiol. 31-SectionB, 31–35 (1940).

096 - Active immunization against pertussis: Final report on the cleveland immunizations of 1934-1935

This is a follow-up to an interim report on a big trial of whooping cough vaccine (065). That report didn't show much difference between vaccinated and unvaccinated, and this final report doesn't improve things much. This is weird, because other big trials at the time that showed pretty positive results (072). There was some suspicion that it was the preparation of the vaccine that made the difference, but that didn't seem to be true (093), so it's pretty mysterious.

So in this final report, the proportions of subjects in each group that got sick were about 15% in the vaccinated and 19% in the unvaccinated controls. Allowing an extra three months before starting to count cases (to let immunity develop after vaccination), the numbers were 16% and 18%, so no better.

The average severity between groups might've been different though. For one thing, there was one death in the control group and none in the vaccinated, but that's not enough to make any conclusions. But apparently multiple people reviewed whatever records they had about severity and came to the same result, that the proportion of mild cases vs. severe cases was higher in the vaccinated group: about 59% mild in vaccinated vs. 35% in controls, and 0% severe vs. 3.5% in controls. But it could've been more rigorous in record-keeping.

So overall, not promising results, but not nothing. Why it differed so much is still mysterious. One interesting thing to note is from this other study that compared several large trials at the time, the attack rate of unvaccinated subjects in Doull's study seemed a lot lower than in others (about half) for some reason.²

Though other reviews seemed to think that this was one of the best studies of the time:

"Doull...took particular care to avoid the numerous pitfalls that attend trials carried out in a clinic population...The same cannot be said of many of the apparently successful trials."³

It'll be interesting to see why the positive studies overwhelmed this kind of result, such that now pertussis vaccine is so common.


References:

1.

Doull, J. et al. Active immunization against pertussis: Final report on the cleveland immunizations of 1934-1935. Am J Dis Child 58, 691–698 (1939).

2.

Perkins, J. E., Stebbins, E. L., Silverman, H. F., Lembcke, P. A. & Blum, B. M. Field Study of the Prophylactic Value of Pertussis Vaccine. Am J Public Health Nations Health 32, 63–72 (1942).

3.

Vaccination against Whooping-cough. BMJ 2, 222–223 (1945).

094 - Active Immunization Against Whooping-Cough

This study has two parts, both relating to whooping cough vaccination. The first seems like a brief summary of a clinical trial, and the second is an animal trial testing which method of killing pertussis bacteria makes the best vaccine.

Part I
This was more exciting, and thus unfortunately lacking in details. There were 288 vaccinated and 1007 control children, followed by questionnaires to parents and physicians. There were 52 known exposures of controls to disease, and 97 exposures in the vaccinated group. From these exposures, 43 resulted in disease in controls, and 10 in vaccinated. This means 82% infection rate in controls, and 10% in vaccinated, meaning 72% protection from the vaccine. That's pretty good.

Also worth mentioning was that 23 of the 97 vaccinated exposures were intimate, meaning siblings that shared a bed or played together or whatever, but none of these resulted in disease in the vaccinated child.

So pretty good, but lacking in necessary details.

Part II
This was a study in mice using vaccines made of killed bacteria, but killed in different ways: phenol, formalin (formaldehyde), merthiolate (thimerosal), or heat. They tried increasing doses of each in mice before challenging with pertussis to see how many died. They found that phenol-killed vaccine required the lowest dose to start showing protection, followed by heat-killed. The others were worse.

The dose required to show any protection, relative to the weight of the mice, was very high, but this could be explained by the requirement that mice be able to resist a serious bacterial infection (the bacteria are injected into their body cavity), rather than a simple respiratory exposure, so it's probably not comparable to humans, in dose at least.

Reference:

Silverthorne, N. Active Immunization Against Whooping-Cough. Can Med Assoc J 41, 263–265 (1939).

092 - Immunization against pertussis

This is another trial of a pertussis vaccine, similar to the others. The justification was that whooping cough is contagious before it is easily recognizable, so it's hard to prevent by anything but immunization.

The subjects were children of 6 to 30 months old, most under 1 year. They were selected from those attending the Stanford Well Baby Clinic randomly, and controls were chosen randomly from the same group, so it was pretty well randomized (except that presumably the parents in the vaccine group had to agree to the vaccine).

The vaccine was whole killed bacteria, a normal dose like Sauer's, 3 doses each a week apart. They started counting after the last dose was given, to make sure everyone was fully immunized. From these shots, they saw 4 systemic reactions, which were just a day of fever.

Then they got the parents to inform them of any known cases of pertussis, and any exposures to others with pertussis. These were diagnosed either by the typical coughing symptom or by culturing the bacteria on cough plates. Some had neither but were counted as "probable" because of history of exposure.

Results
Over the 2.5 years of the study, the vaccinated group had 26 exposures resulting in 7 cases (2 of which were questionable). So the infection rate was 26.9%.

The controls had 25 exposures resulting in 22 cases (1 questionable), so 88% infection rate. The difference between the two groups was statistically significant, both in infection rate and attack rate (the proportion of all subjects infected vs. not), so it seemed like the vaccine was protective.

The control group had 11 cases from unknown exposures, and the vaccinated group had only one, so it's possible to assume the vaccinated group actually had 10 more unknown exposures that would otherwise have resulted in disease, lowering the infection rate to 23%.

In terms of severity, the vaccinated cases were never severe; a few were typical, some mild or questionable. The control cases were mostly typical, 3 severe, 5 mild or questionable. So the vaccine seemed partially protective even in those that got sick.

In terms of immunity duration, of those that got sick in the vaccinated group, most had been vaccinated over 1.5 years before, so the authors recommend small yearly boosters.

Overall, not as rigorous as possible, but pretty good, and the vaccine was helpful but not super-great.

Reference:

Miller, Jr., J. J. & Faber, H. K. Immunization against pertussis. JAMA 112, 1145–1148 (1939).

091 - Active immunization against whooping cough with various specific vaccines

Since various different trials of whooping cough vaccines had given confusing positive and negative results, Morris Siegel wanted to settle the question of whether any version was effective. Sadly, he failed.

This study vaccinated 1324 children in Brooklyn with different versions: Sauer's version from Eli Lilly, a similar version from the NY Department of Health, some versions from Lederle Labs or Povitzky, and then the NY Dept Health version delivered partially subcutaneously instead of all intradermally.

The subjects were under 6, with no history of pertussis, recruited or volunteers from the community. Controls were selected from the same neighborhoods and families. So it wasn't really randomized or placebo-controlled at all.

They followed up once a month with the subjects to get histories of pertussis. Cases were considered cases if whooping was present, or probable cases if there were other symptoms and exposure to a typical case. And cases that happened within a month of vaccination weren't counted either, since immunity hadn't kicked in yet.

So overall there were 1270 vaccinated subjects and 1016 controls that were observed throughout the study. about 80-90% were 1 to 4 years old, and 7-15% were less than a year.

Results
Overall, 3.6% of the vaccinated subjects got whooping cough, vs. 4.2% of the controls. Of the vaccinated, 76% of the cases had received what was considered a full course, three doses.

Not great. However, looking at the breakdown of vaccine versions, no one who got Sauer's vaccine or the similar NY Dept Health one got sick; most of the cases came from the Lederle Labs versions or similar. The intradermal route seemed best. So considering only the good versions, only 1.7% of subjects got sick, vs. 5.4% for all the other versions; worse than the controls. The biggest difference between them is the concentration: there were a lot more dead pertussis bacteria in the seemingly more effective versions.

One other positive thing though was that the age distribution of vaccinated cases was centered on children 2-3 years old, with fewer young infants getting sick than controls, which had a center around 1-2 years. So it's possible the vaccine shifted the age distribution toward older children, who can deal with it better.

Most of the cases happened after four months had passed since vaccination, possibly an indication of waning immunity; or possibly not, since diseases have some seasonality.

Overall, since it wasn't blinded or randomized, and there were so many different versions used in the vaccinated, it's hard to say much about the results. Though some versions seemed to do better than others, it's unclear whether all groups were exposed to the disease equally. So, almost worthless.

Reference:

Siegel, M. Active immunization against whooping cough with various specific vaccines. Am J Dis Child 56, 1294–1303 (1938).

084 - A study in active immunization against pertussis

In the early 20th century, lots of people were working on developing a vaccine against whooping cough. Makes sense, because it was one of the biggest causes of death in young children. But arguably two of the most important of these researchers were Pearl Kendrick and Grace Eldering.

I've done a couple posts about studies by them before (072 and 079); the former of those was a progress report, and this post is the full report on the first large pertussis vaccine trial that they undertook in Grand Rapids, Michigan.¹

The study started in late 1933 and went for 44 months, following thousands of subjects. Soon after the progress report came out, another study (actually another progress report) came out that seemed to have negative results for pertussis vaccination (065), which motivated Kendrick and Eldering to be extra-careful in their own final report.

It was a pretty big effort, not just these two; many nurses and public health workers in Michigan were involved, though supervised by the authors. It took place in Grand Rapids, as I mentioned, and the final count involved 1,815 subjects in the vaccinated group and 2,397 unvaccinated controls. These were children with no history of whooping cough (so, presumably susceptible) that lived nearby and could be followed over the course of the study.

Not all of them remained in the study for the whole 44 months, of course, because once they actually caught pertussis, they were presumably not susceptible anymore. Or if they moved away or something. Or if they grew out of (or into) the age range, which was 8 months old to 5 years old. But the results were corrected for how long each was followed.

The groups were selected by families presenting themselves at clinics to receive the vaccine. These were the vaccinated subjects; others were selected from the same districts as controls. So it wasn't randomized or blinded at all, which is a limitation. Kendrick and Eldering recognized this, and took special care to try to make the groups as equivalent as possible:

-The average time they were part of the study was 15 months for vaccinated, 11.6 months for controls. This was corrected for though, and could be due to some of the controls getting vaccinated and thus being removed from the study, which is something that couldn't happen to those already vaccinated. The proportions that moved away weren't significantly different.
-The proportion of each sex in each group wasn't significantly different.
-The proportions of ages weren't significantly different.
-The proportions in each district of the city weren't significantly different.
-The proportions of family sizes weren't significantly different.
-The proportions of each group getting measles and scarlet fever were equivalent, so it didn't seem like either was healthier or less exposed than the other.
-The average interval between nurse visits for each group was the same.
So overall, the groups seemed equivalent, at least in these characteristics.

As mentioned in 072, the vaccine was made of freshly isolated and lab-cultured bacteria, killed with phenol and/or merthiolate (AKA thimerosal) in small amounts. It was produced continuously on small scales, so none of it got older than about a year. It was injected under the skin of the arms.

In terms of reactions, most of the ones they observed were local—soreness, etc.—and only slight otherwise. One report of the 1815 was of convulsions, and 2 had high fever and vomiting, though it is only correlative because there wasn't a placebo control. Mostly it seemed ok.

Finally, a word on diagnosis and severity ratings: diagnoses were made based on cough plates (that is, culturing the organism), clinical symptoms, and history of exposure. Severity was rated somewhat arbitrarily, based on frequency of whooping and vomiting or the occurrence of complications/weight loss.

Results
The results were corrected for amount of time each subject was participating in the study, and how many subjects there were in each group, so they're reported as annual attacks per 100 subjects. So with that in mind, the incidence of whooping cough overall was:
2.3 annual attacks per 100 vaccinated subjects
vs.
15.1 annual attacks per 100 controls.
This is a significant difference, which could be expected to occur by random chance only once if they repeated the trial millions of times.

Compared to other reports of the rates of pertussis in Grand Rapids, the control group followed the same up-and-down trends, but had a higher incidence, probably because the closer observation detected cases that would've gone undetected otherwise, due to low severity. And speaking of severity:

Even of the vaccinated subjects that did get whooping cough, the severity was much lower. 73% were rated as light or very light severity, compared to 27% in the controls. And only 4% were severe in the vaccinated, and these didn't have serious complications, only frequent coughing/vomiting; that's compared to 13% severe in the controls. And considering that the "very light" cases were questionable about whether they could even be considered cases at all, by removing them from both groups, the difference in incidence increases even more.

Kendrick and Eldering also looked specifically at their data on known exposures of subjects to the disease, and found similar patterns. The vaccinated group actually had a higher number of exposures than the controls, but many fewer cases from them. Calculating the number of cases expected (based on the cases in the controls and the exposures in the vaccinated), it appears that the vaccine prevented about 81.3% of cases in the test subjects. Not excellent, but pretty good.

The differences were not as big when the exposures were more intimate, like within a household. The attack rates were about 35% vaccinated vs. 90% controls. Still, that's significant protection.

Another thing they controlled when watching exposures was coughs that weren't diagnosed as pertussis, of which there were more in the vaccinated group. These were mild, taking place right around an exposure incident, and could be classified as slightly less than "Very light" whooping cough. So if they added these coughs to the numbers, incidences overall would be 24% vaccinated vs. 72% controls, which means only 67% protection. Which is still pretty good, especially considering that these extra cases are barely cases at all.

So overall, this study is very good compared to others at the time or before, though not quite good judging by modern standards: not randomized, not placebo-controlled, though more controlled than it could've been. And the protection seemed pretty good, though arguably it would be better to have higher complete protection rather than just reducing the severity of cases.
It isn't a study that looked at the duration of immunity, or whether it reduced transmission at all (thus providing herd immunity), or if the vaccine was safe, especially in the long run, or if it could be effective in more than just this population. But what it does look at is how well it protects young children over at least a few years after it's given.

Oh, and it's not funded by any pharmaceutical company.

It's hard to make any judgments overall, so I'll just cite some comments from later publications that cited this one, both positive and negative:

"There is ample evidence in the literature now that individuals inoculated with suitable doses of a proper vaccine have a high degree of immunity against whooping cough."²

"The Sargent-Merrell method of evaluating the success of an immunization program has been applied to data covering a 6-year period in the city of Grand Rapids...the proportion of cases prevented is 84%. The validity of the result has been verified on the basis of controlled field data."³

"The controversy dates back to the first trials of pertussis vaccines, which were carried out during the 1930s. These were criticized as biased in favor of the vaccines because they were not randomized; vaccinated volunteers were compared with unvaccinated 'nonvolunteers.'"⁴

"Methodologically, the original field trial design was flawed. The experimental group was self-selected and only control subjects were randomly chosen. Despite careful attention paid to case detection and diagnosis, 1603 observations from the study's early years had to be excluded from the final analysis. Several featuers of the trial nonetheless make it an important contribution, not simply to the development of an effective pertussis vaccine, but to the history of controlled trials: ... 2) The trial was unusual for the level of attention given to case diagnosis and follow-up, and to the discussion of unknown factors which might have biased the results; 3) a similar level of detail was given in reporting the analysis and the methodological limitations of the field trial"⁶

Also, if you want a really detailed historical account of this study and everything that went into it, before and after, check out Shapiro-Shapin 2007.⁵

References:

1.

Kendrick, P. & Eldering, G. A study in active immunization against pertussis. Am. J. Hyg. 29, 133–153 (1939).

2.

Cohen, P. & Scadron, S. J. The placental transmission of protective antibodies against whooping cough: By inoculation of the pregnant mother. JAMA 121, 656–662 (1943).

3.

Weiss, E. S. & Kendrick, P. L. The Effectiveness of Pertussis Vaccine: An Application of Sargent and Merrell’s Method of Measurement. Am. J. Epidemiol. 38, 306–309 (1943).

4.

Fine, P. E. M. & Clarkson, J. A. Reflections on the Efficacy of Pertussis Vaccines. Reviews of Infectious Diseases 9, 866–883 (1987).

5.

Shapiro-Shapin, C. G. ‘A Whole Community Working Together’: Pearl Kendrick, Grace Eldering, and the Grand Rapids Pertussis Trials, 1932-1939. Michigan Historical Review 33, 59–85 (2007).

6.

Marks, H. M. The Kendrick-Eldering-(Frost) pertussis vaccine field trial. J R Soc Med 100, 242–247 (2007).

072 - Progress Report on Pertussis Immunization

Other than Louis Sauer and Madsen and others, Pearl Kendrick and Grace Eldering were trying to develop a good pertussis vaccine. They did a trial of the one they made in Grand Rapids, Michigan, which from what I can tell was pretty historic. This report was not their final conclusion, but they had enough data to make an interesting preliminary report.¹

They used a virulent strain of B. pertussis for the vaccine, growing it on sheep’s blood, then killing it with merthiolate (AKA thiomersal or thimerosal) or phenol. In terms of dose, they used closer to Sauer’s preferred dose, much higher than Madsen’s, but tried a few different doses (though this didn’t seem to make much difference).

The subjects in the study were children, 8 months to 5 years old, especially 1-2 years. As controls, they followed children the same age, preferring children in the same families as those who received the vaccine (to ensure similar circumstances).

They worked with nurses from the city’s public health department to follow up with the subjects at 3- or 4-month intervals, verifying reports of pertussis cases or exposures. Exposures were classified as definite (coming from someone in the home), indefinite (coming from elsewhere), or unknown (a case of whooping cough from an unknown source). Cases were rated by severity based on quantitative definitions of duration of disease, cough, whether there were complications, etc.

So what did they find? It was pretty good. By the time of publication, sixty vaccinated subjects had been exposed, but only four got sick. That’s less than 7% (or 0.5% of the total group, including non-exposed subjects, 712 subjects total).

In contrast, there were 63 cases in the control group, from 84 exposures (880 subjects total). That’s 75% infection rate (or 7% overall). So the vaccine seemed to protect more than 90%.

And even considering the four vaccinated subjects that got sick, their disease was either light or very light, while 85% of the control cases were of medium severity or higher.

The researchers warn that the numbers might be too small to make any solid conclusions, and the trial is not up to today’s standards (in terms of blinding, randomization, placebos, etc) but for what it’s worth it seems impressive.

A later historical account gives the following description of the response to this report. Keep in mind that requirements for trials in the 1930s were much less rigorous than they are now, and that yet Frost basically admits he can’t find much fault with the design: 

"Although it was well controlled, the 1934-1936 field trial had to surmount considerable initial skepticism within the national public health community. Shortly after Kendrick and Eldering announced their results, James Doull, a prominent Cleveland epidemiologist, reported that children received no protection from a vaccine he had designed and produced (065). The APHA subcommittee on whooping cough, which included both Kendrick and Doull, evaluated the contradictory results of the two studies but was unable to explain why they differed. The committee then asked Wade Hampton Frost, a Johns Hopkins epidemiologist and head of the APHA, to review both studies.
     Frost was predisposed to find fault with Kendrick's study. In a "personal" note...Frost voiced his doubts: 'I very strongly suspect that Miss Kendricks field studies are not set up in such a way as to give a really good control. My reason for this suspicion is that, as you know well enough, the satisfactory set-up of such an experiment is an exceedingly difficult matter. Not 1 out of 10—perhaps not 1 out of 50—attempts is successful and as a mere matter of probability the odds are strongly against Miss Kendrick's experiments being sound.'
     Unable to find the faults he expected in Kendrick and Eldering's study, Frost journeyed twice to Grand Rapids to examine their data and recommend suggestions for improving statistical accuracy and coding. In the end, Frost said to Kendrick, 'I think it may be assumed, not as a conclusion but merely as a working hypothesis, that your data when finally analyzed are likely to show some protection in the vaccinated group.'"²

Citations:

1.  Kendrick, P. & Eldering, G. Progress Report on Pertussis Immunization. Am J Public Health Nations Health 26, 8–12 (1936).

2.  Shapiro-Shapin, C. G. ‘A Whole Community Working Together’: Pearl Kendrick, Grace Eldering, and the Grand Rapids Pertussis Trials, 1932-1939. Michigan Historical Review 33, 59–85 (2007).

070 - Immunization with Bacillus pertussis vaccine

As today's article states near its beginning:

"Most whooping cough patients need no physician, but those who do, need him badly." -Pospischill

Louis Sauer was one of the first to develop a whooping cough vaccine that actually seemed to work, at least sometimes. This was another article giving some data of efficacy.¹

As usual, the bacteria to make it were from fresh isolates of the disease, grown on human blood and suspended/killed in saline plus a little phenol.

Sauer first determined that the vaccine didn't work for vaccine therapy (i.e. using a vaccine to treat a disease, rather than prevent it). Not surprising; that concept didn't work too well in general.

So the next test would be for prophylaxis, preventing the disease by giving the vaccine to people before they were exposed to the disease. So he injected it into 394 children, most of whom had an older sibling known to be likely susceptible to pertussis (i.e. they hadn't been vaccinated or caught the disease before); these siblings were the controls. He used 3 injections total; he tried a single larger dose for some, but it seemed more prone to side effects.

Sauer reported that he did warn parents of the children about potential side effects, such as temporary fever and reactions at the site of injection, but all of them were eager for the treatment except for two, who refused it.

So within these children, over the course of 5 years, there were about 191 exposures to whooping cough, but not a single vaccinated subject got the disease. Of the controls, 31 got it, including many who were siblings of the vaccinated. There was definitely a lot of exposure of the vaccinated from their siblings, including as intimate as kissing (platonic, of course).

So Sauer concluded that his vaccine produced immunity after four months, at most, which lasted at least a few years. He recommended vaccination at ages 6-12 months, because immunity could definitely be produced at that age, and pertussis is most dangerous within the first two years.

This study had problems, of course, and isn't up to modern standards at all, but the results are fairly striking.

There was some discussion between physicians published after the main article, some of which was related and interesting:

"Dr. Sauer seems to have found a method of preparation and a method of administration of pertussis vaccine that will protect a child from whooping cough just as certainly as toxoid protects against diphtheria. His success in immunizing against pertussis is probably due to two factors. The first factor is the preparation of the vaccine. He uses only fresh cultures obtained every few months from active proved cases of pertussis. The second contributing factor is the tremendous dosage." -Dr. Raymond Schowalter

"Three or four months should elapse or intervene between completion of the vaccine administration and exposure to the disease. During the five years that this work has been going on, people wanted their children injected while whooping cough was in their neighborhoods. This was done and in a number of cases, within one or two months after the injections were begun, these children contracted pertussis. But in no case in which the time interval was more than three months has any child contracted pertussis, although, as the charts show, eighteen were intimately exposed to control cases in the family." -Louis Sauer 

 Also a later publication made an interesting remark on the quality of this study:

"Sauer's reports since 1933 as to the prophylactic value of [his vaccine] seem convincing, except for the fact that he does not fully discuss his results in an adequate number of control children."²

References:
1. Sauer, L. Immunization with Bacillus pertussis vaccine. JAMA 101, 1449–1453 (1933).
2. Siegel, M. & Goldberger, E. W. Active immunization of tuberculous children against whooping cough with Sauer’s vaccine. JAMA 109, 1088–1092 (1937).

069 - Vaccination against whooping cough

More about whooping cough. In Denmark, the State Serum Institute (a kind of non-profit government-run lab for making immunity-related products, I think) in Copenhagen made a vaccine against whooping cough, using recently isolated strains of the bacteria, killed and washed and suspended in saline with 1% formaldehyde. These were given in 3 injections, 3-4 days between them.

In this study, Thorvald Madsen discusses the results of vaccination in two whooping cough epidemics in the Faroe Islands using this vaccine.¹ In his experience, the vaccine was a good one because it didn’t cause many bad reactions, though he noted that a couple young infants died within 2 days of receiving it; any connection to the vaccine was unclear, so it could easily have been coincidence, no way to know. Still, he recommended against the vaccine for children under 1 month old.

The Faroe Islands, off the coast of Denmark, were pretty isolated, at least at that time, so contagious diseases often came in waves. After one outbreak, a while would pass before a second, and then the disease would break out and spread through the population again.

In the outbreak just at the beginning of the study, lots of people had just been vaccinated, and many others hadn’t, so there was a good comparison. Apparently the vaccine had come too late, so nearly everyone caught the disease, vaccinated or not. However, the disease appeared to be much milder in the vaccinated group: only 5 people died of 2094 vaccinated, compared to 18 deaths in 627 unvaccinated. That’s twelve times fewer in the vaccinated. The disease was reportedly more severe in the unvaccinated too, even those that survived. Best results were seen in those who had been vaccinated at least a week before the outbreak.

Then in 1929 there was another epidemic. 1832 people had been vaccinated, and 446 had not. Of the vaccinated, 458 were disease-free! And of the ones that weren’t, most of their disease was mild; only 1 died. Of the unvaccinated, only 8 had no disease, and of the rest only half had mild disease; the rest was more severe. 8 died. So the mortality in the vaccinated was 1/16th that of the unvaccinated. The vaccination had been done at the right time. These were pretty good results.

Combining the results from the epidemics, the vaccinated mortality rate was 0.15%, vs. the unvaccinated mortality rate of 2.4%. A big difference, especially combined with milder disease in general.

Madsen says the subjects were comparable in terms of age, time of epidemic, and other factors between the vaccinated and unvaccinated groups, and the ones vaccinated were chosen pretty much randomly.

Other scientists studying whooping cough vaccines at the time acknowledge the positive results, and point out differences between Madsen’s vaccine and other contemporary version:

"The chief difference between Madsen's and [Sauer's and ours] is in dosage, Madsen using a total of only 2 c.c. as compared with the 7 or 8 c.c. given by the others. The other differences are in kind of blood used to enrich the medium, in certain details of seeding and harvesting, and in choice of a killing agent. There is no experimental proof that these differences have any marked effect on the final product."²

Another article complains (fairly) about how little information there is in Madsen’s account:

"Madsen says practically nothing about his two groups except that they were unselected and were comparable in respect of age, time of epidemic, and surrounding conditions."³

This is true, it wouldn’t really pass muster as a modern study. There was no placebo or blinding either. However, while many people have cited this study as the first evidence of harmful side effects from a pertussis vaccine, the same complaints may apply:

"Madsen, it is true, recorded two examples of fatal convulsions after injection of fluid vaccine in newly born infants, but it seemed likely that age was the precipitating factor."⁴

"Although there have been reports of neurological illness after pertussis immunisation since 1933, none has been based on established epidemiological methods using relevant controls."⁵

In summary, it seems like low-quality but positive results in favor of vaccination, and at most a warning of something to watch for as an adverse effect in future studies.

References:

1.  Madsen, T. Vaccination against whooping cough. JAMA 101, 187–188 (1933).

2.  Eldering, G. & Kendrick, P. L. Some Practical Considerations in B. pertussis Vaccine Preparation. Am J Public Health Nations Health 26, 506–511 (1936).

3.  Vaccination against Whooping-cough. BMJ 2, 222–223 (1945).

4.  Pertussis Vaccination and Encephalopathy. BMJ 1, 110–112 (1950).

5.  Miller, D. L., Ross, E. M., Alderslade, R., Bellman, M. H. & Rawson, N. S. Pertussis immunisation and serious acute neurological illness in children. BMJ 282, 1595–1599 (1981).

067 - Active immunization of tuberculous children against whooping cough with Sauer's vaccine

It seems to be the age of whooping cough vaccine testing, at least in this blog. This is another.

Given other questionable results (065), though not with Sauer's vaccine exactly, Siegel and Goldberger decided to do a more controlled trial, in the Sea View Hospital for people with tuberculosis. This place had a bunch of children with TB that were kept together by age and sex (after a certain age), with limited contact with the outside world. Since they were monitored and isolated, pertussis would be easier to track among them.

So Siegel and Goldberger got vaccine from Sauer and Eli Lilly, and vaccinated a total of 101 children over 2 years, keeping a second group as controls. They stopped vaccinating when an outbreak of pertussis occurred in the hospital. Sixty-four children in total were exposed over 3.5 months, and there were 27 definite cases of characteristic whooping cough. The staff isolated each child when they started having symptoms of course, but symptoms don't show up until after the child starts being able to spread the bacteria to their playmates. Another 5 children got sick but didn't have the characteristic whoop, so they were counted as probable cases.

At the time of the outbreak, the average age of vaccinated children was 4.2 years, vs. 2.3 years average for controls. This is a big difference, and could affect the results. The groups didn't have significantly different severity of tuberculosis though.

There were only 36 children in the final study, too: 17 vaccinated and 19 controls. The others either had left the hospital, or had certainly or possibly encountered pertussis previously (and thus maybe had some natural immunity). They only considered those with no known history.

So now for results: considering only definite cases, vaccinated did better, 29% (5 of 17) getting sick, vs. controls with 53% (10 of 19). 44% decrease. Not very good at all. Especially when adding in probable cases, which brought the numbers up to 53% vs. 58%.

Most of these cases were mild; there was one moderately severe case among the vaccinated, and four more in the controls. Pertussis didn't seem to aggravate TB in either group. The average durations of whoop and severe period in the vaccinated group were 21 and 2.8 days, vs. 32.5 and 7.5 days in controls. And considering more of the cases in the vaccinated group weren't even certainly whooping cough, it seemed like the disease was less severe in those vaccinated. However, these children were older too, so it's possible the age might've contributed more than the vaccine to this effect.

So these aren't great results for the vaccine, especially considering the lack of blinding or placebo. It's possible some things complicated the results: having tuberculosis may have increased the children's susceptibility, but according to studies of antibodies, they seemed to respond to the vaccine as well as normal children do. Also, being so closely intimate in the hospital setting, there might've been just too heavy an exposure to the pathogen for the vaccine to be adequate. But it seems like if it were a good vaccine, that shouldn't be an issue. So, more work to do perhaps.

Citation: Siegel, M. & Goldberger, E. W. Active immunization of tuberculous children against whooping cough with Sauer’s vaccine. JAMA 109, 1088–1092 (1937).

065 - Active Immunization Against Whooping Cough: Interim Report of the Cleveland Experience

Apparently there was a lot of interest in a vaccine against whooping cough around this time. Makes sense; it sounded pretty serious. People had recently discovered that when growing the pertussis bacteria in the lab, they started off with smooth-looking colonies that were virulent, which over time sometimes got more rough and less harmful. Some graded these from Phase I (smoothest, most virulent) to Phase IV (roughest, least virulent). Sauer and others thought the Phase I would make the best vaccine.

Studying vaccines of pertussis was tricky: the way the vaccine was made could influence things, obviously (as this study itself seems to show), and diagnosing the disease could be uncertain, and they weren’t even sure if having the bacteria in your body was even enough to make you sick; some might have the bacteria and not be sick, though there seemed to be some evidence against that idea.

Based on previous experience, James Doull, Gerald Shibley, and Joseph McClelland decided that a study of 200-300 children in each group (vaccinated and unvaccinated) could detect a difference in pertussis cases decently. But just to be safe, they decided to go with a higher number, around 500 per group.¹

They recruited these children at health stations in Cleveland that gave out free milk. The requirements were that the children must not have had pertussis before, and needed to have an older sibling who also had not had pertussis (to increase the likelihood of exposure), and had to be between 6-15 months old.

They started out planning to give every other child the vaccine they made, but a lot of parents refused to let their children be vaccinated, so they ended up counting some refusers as controls instead. So there ended up being 483 vaccinated (most with all three doses, a few with only one or two), 247 selected as controls, and 249 that refused. So almost 1000 total.

The vaccine was made with 5 recently isolated (and thus smooth and virulent) strains of bacteria, grown on agar, scraped off, washed with distilled water, and suspended in saline with 0.5% phenol. Standardized doses were given once a week for three weeks, subcutaneously in the buttocks. Sounds like fun.

They noticed some reactions to it, but not many. Some had a slight fever on the same day, or a local reaction lasting up to 3 days. It was claimed that 3 had convulsions, one of which was observed by a nurse.

After inoculation, the authors kept tabs on their subjects, sometimes sending someone to check on the families. If any reported a case, an epidemiologist or pediatrician obtained records and tests to confirm.

The distribution of race and sex in the groups seemed pretty even, though there might’ve been more white people refusing the vaccine than minorities. The age distribution was fairly even though too.

Ok, results: not great. 61 out of 483 vaccinated got whooping cough, or 12.6%. Of those not vaccinated, either from randomness or refusal, 71 of 496 (14.3%) got whooping cough. So at most, the vaccine was 11.6% effective, or prevented the disease in 11.6% of those who would’ve gotten it. Pretty awful.

They did mention that some thought the vaccinated cases might’ve been milder, but it seemed like a rather subjective judgment:

"The opinion of physicians who have seen representative attacks in both groups is that those in the inoculated have been milder. This is a difficult question to settle. There has been only 1 death, and that in a control child."

Overall, seems like pretty clear negative results. It wasn’t a great study (no placebo, no blinding, not even very good randomization), but usually one expects those problems to give more positive results, not less positive. So it’s somewhat interesting, considering the positive results others had observed with similar vaccines around that time.

Some people in later articles had some ideas about what the difference might be:

"In three field studies...favorable results were reported. In another very thorough study [this one] in which the method of preparation of vaccine was slightly different unequivocal evidence of immunization was not obtained."²

"The Cleveland vaccine [this study] was also a Phase I vaccine [freshly isolated organisms] containing 10 billion organisms per ml., but in its preparation the organisms had been washed once with distilled water, while the Michigan vaccine [showing a positive result] had been washed once with saline."³

"These varying estimates of vaccine performance had at least two possible explanations. They might have reflected either differing manufacturing processes, or variations in the methodological rigour of the studies."⁴

"In his pertussis studies, Doull relied on alternation, which in his view was sufficient to prevent bias in selecting which children would receive the vaccine. However, the actual allocations departed from this 'ideal' somewhat: in one recruiting location, Doull used children whose parents refused inoculation as the controls, while at a later point in the study, he assigned the vaccine in a 2:1 ratio, with every third child serving as a control."⁵

Citations:

1.  Doull, J. A., Shibley, G. S. & McClelland, J. E. Active Immunization Against Whooping Cough: Interim Report of the Cleveland Experience. Am J Public Health Nations Health 26, 1097 (1936).

2.  Singer-Brooks, C. & Miller, J. J. The Opsono-Cytophagic Test in Children with Pertussis and in Children Vaccinated with H. pertussis Antigens. J Clin Invest 16, 749–761 (1937).

3.  Perkins, J. E., Stebbins, E. L., Silverman, H. F., Lembcke, P. A. & Blum, B. M. Field Study of the Prophylactic Value of Pertussis Vaccine. Am J Public Health Nations Health 32, 63–72 (1942).

4.  Jefferson, T. Why the MRC randomized trials of whooping cough (pertussis) vaccines remain important more than half a century after they were done. J R Soc Med 100, 343–345 (2007).

5.  Marks, H. M. James Angus Doull and the well-controlled common cold. J R Soc Med 101, 517–519 (2008).

064 - Prophylactic pertussis immunization

Today's study is another test of whooping cough vaccines. Sauer's was the popular one at this time, showing 92% efficacy in some studies, but a Lucy Mishulow had created one that used a stock strain of pertussis instead of needing to isolate a new one with every outbreak, as the Sauer version called for. Also, it could be grown on animal blood instead of human blood. This made it more potentially useful.

So Eli Shorr selected preschool children with known histories of illness to vaccine or keep as controls. He tried to divide families up evenly between vaccinated and controls, to make things as equivalent as possible. Half the subjects attended the same nursery school.

They compared different methods of inoculation: intramuscular, intracutaneous, and subcutaneous. After, they looked at levels of agglutinins in the blood as a measure of antibody response.

In order to compare, they gave a few children the Sauer vaccine, and some of the controls got an injection of sterile diluted milk as a placebo.

They found that about 36% of the subjects were definitely exposed to pertussis over the next 30 months; probably more that they didn't notice. 16% of those receiving Mishulow's vaccine got pertussis, 10 out of 63, though 6 of the 10 were very mild cases (no whooping even), and also 6 of the 10 got sick within the period Sauer thought might be too soon for immunity to develop, and 4 of them had smaller doses of vaccine too. So it might be said that only 10% of well-vaccinated subjects got sick.

With the Sauer vaccine, 1 of 11 (9%) got sick, but that one had gotten a low dose too.

Whereas with controls, 26 of 72 (36%) got pertussis. Big difference. And they weren't even observed for as long a period.

In terms of inoculation route, intracutaneous was abandoned early because it was difficult to give an adequate dose and reactions to it were overly severe. Reactions to the other routes were never severe; most just had slight tenderness, some had redness or a low fever for a day or two.

The antibody measurements were as expected, pretty high soon after vaccination but none or very low in unvaccinated group. Levels fell after 2-3 months, but it wasn't clear if that meant immunity went away or anything.

So overall, it seemed like Mishulow's vaccine was pretty good, but it was too small a study to compare to Sauer's very well. Googling "Lucy Mishulow" doesn't bring up many results, so I wonder if the vaccine turned out badly somehow, such that this vaccine researcher got mostly lost to the sands of time somehow. Maybe I'll find out later.

Citation: Shorr, E. Y. Prophylactic pertussis immunization. The Journal of Pediatrics 9, 49–55 (1936).

061 - Experimental Pertussis

This is a pretty neat study, though it kinda makes me cringe. You’ll see why.

Whooping cough is a respiratory infection caused by the bacterium Bordetella pertussis, though apparently there was a lot of debate about the pathogen (bacteria or virus?) in the first half of last century. This study was designed in part to test that.

The other part was testing whether Louis Sauer’s vaccine made from B. pertussis could protect against the disease (which would be another indication of its bacterial cause). So how did it go?

H. and E.J. Macdonald, a physician husband and nurse wife team, intentionally exposed four healthy brothers aged 6 to 9 years to cultures from a separate whooping cough patient. These boys, it turns out, were their own sons. Now that’s dedication to science!¹

Two of the boys, the 9-year-old and one of the 8-year-old twins, had been vaccinated by Sauer 5 months before, and the other two (8 and 6 years) had not. None had any previous exposure to pertussis.

The team took a cough plate culture from someone with typical whooping cough and grew cultures from it on agar, checking under a microscope to make sure it was a pure culture. Half of the growth on this plate they suspended in saline solution, and then filtered it through a filter with pores small enough to remove bacteria from the solution, presumably leaving only viruses, if there were any. The other half of the growth they suspended in saline without filtering.

To start, they squirted a little of the filtered solution into the boys’ nose and throat, then quarantined them in a rural apartment with their mother (the nurse) for 8 weeks. They didn’t come down with any symptoms within 18 days, long enough for whooping cough to show up, so it didn’t seem to be some virus present in the culture.

So then after the 18 days, the team squirted some of the unfiltered suspension into the boys’ nose and throat. They aimed for about 140 bacteria total per boy. First the vaccinated results: neither of the two vaccinated boys had any symptoms or sign of whooping cough in the whole period of 38 days. Cultures from their throats and such were consistently negative.

On the other hand, the unvaccinated boys started coughing after only 7 days. Cultures were rated as ++++, which seems very positive, even from the beginning. Over the next few weeks, their fever and coughing increased in severity, they started whooping and vomiting food and mucus, stopped eating much, and had headaches. Seems pretty miserable. Then they got better, fortunately.

After recovering, the team tested the antibodies of all four boys, as well as two others each that were known to be immune or non-immune, and found that all were positive except the two known non-immunes.

So what could be concluded from this: as few as 140 cells is enough to cause an infection. B. pertussis is the agent that causes whooping cough. Seven days is the incubation period (at least here). Possibly also that the vaccine works pretty well.

On the other hand, it’s definitely a small sample size (2 patients in each group), and there was no blinding or placebo, but it gave very distinct results in a very controlled situation. All of them were known to have been exposed to enough pathogen to cause disease, and none could’ve been exposed from somewhere else. The populations were pretty matched too: two of the boys were twins, one vaccinated and one not. But one could argue that it’s not good enough.

As a minor question, I’m not even sure why they would’ve thought there would be any virus on the culture plate, unless they thought it were stuck to and replicating along with the bacteria or something…

And finally, the cringe-y part: this seems so unethical based on my understanding of standards for medical research these days, exposing children to a potentially deadly disease, but at least we can benefit somewhat from the results.

Some others agree with me in some ways and make observations:

"In...1933 the Macdonald husband-and-wife team performed an experiment on their four sons, from which they concluded that 'a filter-passing virus plays no role in the etiology of pertussis.' The wife, a nurse, sequestered herself with the boys in a rural apartment for eight weeks...Aside from proving that there are hazards in being born into a physician's family, and that B. pertussis could cause whooping cough, the findings did not really exclude the possibility of a direct or indirect role for viruses in the disease. It would have been a hardy virus to survive through two subcultures on agar medium."² [Though later studies confirm the result.]

"In 1933, Sauer vaccinated 2 of 4 brothers; all 4 brothers were then inoculated in the nose and throat with whooping cough bacillus. The 2 hapless controls (sons of a local physician) developed classic cases of whooping cough while their vaccinated siblings remained healthy."³

The four boys.
Source: National Library of Medicine, and Baker 2000³

Citations:

1. MacDonald, H. & MacDonald, E. J. Experimental Pertussis. The Journal of Infectious Diseases 53, 328–330 (1933).

2. Nelson, J. D. Whooping Cough — Viral or Bacterial Disease? New England Journal of Medicine 283, 428–429 (1970).

3. Baker, J. P. Immunization and the American Way: 4 Childhood Vaccines. American Journal of Public Health 90, 199 (2000).

046 - Results of Field Studies With Poliomyelitis Vaccine

One of the vaccines I'm excited to learn about is polio, because apparently it works well enough that we've almost eradicated it from the world. But apparently it took a while to develop, probably because it took a while to figure out what caused it. (Something way smaller than bacteria? Whoa)

But in today's study, Gilliam and Onstott do a pretty good trial to test a vaccine against polio!¹ This version was developed by someone named Brodie, so it's not one of the ones in use today (which are Salk and Sabin), but I wonder why not. Apparently it was made of an emulsion of virus from monkey spinal cords, treated with formaldehyde long enough to inactivate the virus.² It seemed safe enough in preliminary tests, so the authors went ahead with a field trial.

They selected several communities in North Carolina and Virginia, near where polio outbreaks were occurring but not near enough to be in the midst of an outbreak (they wanted to make sure there was enough time to establish immunity before exposure). To parents of young children in these communities, they offered the vaccine as experimental and reasonably safe, and got 1452 volunteers for the trial in total.

Of these, they split them into those who would receive the vaccine, 766, and the control group, 686, based on the place of their surnames in the alphabet. Nice and random, supposedly. Each patient's physician would be the one doing the vaccination though.

Brodie cooperated with the study by providing the vaccine and also by withholding it from anyone else in the area who might want it, so that the researchers could be sure the controls really were controls.

When it came time to actually get the shots, only 458 volunteers actually showed up; a few of the controls managed to come upon a vaccine in other ways too, so they were excluded from the study.

There were two doses of vaccine, about two weeks apart. At first, they tried giving a shot of novocaine along with the vaccine to make it less painful, but it seemed just to complicate things without helping much. So in the end, 422 subjects got both doses, mostly at the right interval, and 36 got only one.

Gilliam and Onstott were closely involved in following up with the patients, asking about side effects or symptoms they may have had, since apparently the physicians missed some things. They observed that 50% of the subjects had some sort of reaction, though mostly mild and temporary: redness, swelling, etc. Most of the reactions seemed to be in patients that had received the novocaine. Hmmmm.

However, there were 15 local reactions that were more serious. Fourteen patients had an abscess or similar at the injection site, all of which healed, sometimes slowly.

Seventeen patients had general reactions (like fever or headache or nausea), which isn't bad, and these were usually associated with the abscesses. But some of the reactions were disturbing. For example, one girl fainted and stopped breathing after the shot, though she was fine after they revived her. Another had leg cramps at the injection site, though they got less frequent over time.

So it's pretty clear that this vaccine was not as safe as they thought, or at least not as safe as it should be. Better than being vulnerable to polio? Perhaps, but still not ideal.

What were the results other than safety? Well, of people who weren't involved in the study at all, about 6 in 10000 had polio, so the number of people you'd expect to have the disease in a similar group of a thousand is about 1, plus or minus 1. So between zero and two. In actuality, they didn't see any cases in either the vaccinated or controls. The authors calculated that even if the vaccine were 100% effective, they'd have to have about 10000 patients in each group to see a significant difference between them.

So that's unfortunate; the study was pretty much useless for judging effectiveness. As a later publication says,

"They were able to show, however, that Brodie's vaccine was relatively harmless, although they encountered a few very disturbing reactions."²

I had some other issues with the design of the trial as well. First, a minor issue: randomizing by alphabet? I don't think it's safe to assume that there isn't some bias inherent in people's family names, such that a group of people with names from the first half of the alphabet would be effectively identical to a group of people with names from the second half. But maybe.

Second, there's no placebo, which means there's almost no blinding (everyone knows who gets the vaccine and who doesn't), so bias in diagnosis could be introduced (like assuming people who got the vaccine are less likely to get polio, so being more likely to attribute symptoms to some other disease).
It also means that people dropping out of the study may not have been random; certain types of people might drop out more often in the vaccinated group but not in the control group (because there's no shot or other commitment involved), so the groups wouldn't be equivalent.

It didn't seem like there was much monitoring of rates of adverse events in the controls, to see if the observed side effects could be reasonably attributed to the vaccine or not, but perhaps that would need a different kind of study.

So overall, it's not great.

Citations:

1.

Gilliam, A. G. & Onstott, R. H. Results of Field Studies With Poliomyelitis Vaccine. Am J Public Health Nations Health 26, 113–118 (1936).

2.

Maxcy, K. F. Changing Conceptions of the Prevention of Acute Anterior Poliomyelitis. The American Journal of the Medical Sciences 192, 436–444 (1936).