List A:  The Hallmarks of Science Research

 

These are things I expect in an article, video, or lecture that indicate the author/presenter is investigating the topic and trying to learn with Honesty, Integrity, Rigor and Humility.

​

1. Actually does some work that could lead to an increase in our understanding of something (Doesn’t just criticize the work of others, but identifies something they want to learn about and comes up with some way to investigate it, does an experiment, etc).

2. Thoroughly explains the tools, methods and calculations they used

3. Explains all assumptions they made and discuss why they are reasonable and/or necessary

4. For any statements that are not simple and obvious, they provide a detailed reference to a source of that information so a reader can go read it to check it or to learn more

5. Has had their work pass peer review by other experts chosen for their knowledge of the subject, with no regard to their personal religious, political or other views irrelevant to the scientific subject

6. Publishes to an audience of experts, who will be able to give the work the highest scrutiny

7. Promptly corrects any errors that someone points out after publication

8. Explains any limitations of their work

9. Explains the degree of confidence in the results based on the methods used, using math when possible

10. Provides a way for people to point out errors and a way for the errors to be corrected

11. Clearly defines all terms used

12. Works in an environment where they would face severe consequences if found to be misrepresenting someone’s else’s work or fudging anything.

13. Is familiar with the work of other experts in the field and discusses whether or not their own results match those of others

14. When referring to the work of others, accurately represents the other’s position or results on the subject. 

15. Does everything they can to check their results for accuracy

16. Understands that people can have both obvious and hidden biases and takes steps to eliminate or minimize them

​

​

List B:  The Hallmarks of Pseudoscience and Propaganda

​

These are red flags I look for in an article, video, or lecture indicating the author/presenter is just trying to convince me to believe as they believe, with little regard for Honesty, Integrity, Rigor and Humility

 

1. Doesn’t do any actual work of their own that could increase our understanding of the subject, but uses the subject only to promote a philosophy, belief, political view, etc.  (Does no field work, no lab work, no development of new techniques or methods, etc)

2. Misrepresents someone else’s view in order to make their own view seems more reasonable (strawmanning)

3. Appears to think that quotes from people are meaningful as evidence

4. Misquotes others or quotes them out of context to make the quote appear to support a certain claim  (quote mining)

5. Has either not subjected their work to peer review at all, or only to review by others who hold the same personal religious / political / etc. views

6. Gives two conflicting answers to the same question in different places, depending on what particular point they are trying to support at the moment

7. Publishes to an audience that is not knowledgeable on the subject, and thus is easy to convince

8. Holds a personal commitment to one specific answer to the question they are discussing

9. Claims that people who disagree are doing so because of some ulterior motive, without providing evidence of that

10. Expects you to just believe them or someone they quote based on their perceived authority

11. Fails to define important terms uses

12. Uses the same word for two things or ideas that are not the same, either due to lack of rigor or intentionally

13. Uses faulty logic (logical fallacies)

14. Refers to another source in support of their point, but the source doesn’t say what the person is claiming it says

15. Refers to another source in support of their point, but the source actually contradicts their point

16. Cherry picks data/information from sources to make it falsely appear to support their view

17. Doesn’t mention other important work that has been done in the field whose results do not agree, either due to being unaware or due to intentionally leaving it out

18. Appeals to my emotions as a means to get me to accept their claim

19. Threatens me with consequences if I don’t accept what they say

20. Does not correct errors that are pointed out to them

21. Repeats something they heard from a website, blog, or Youtube video without checking it

22. Instead of providing any meaningful evidence for their claims, they adopt a victim mentality or claim others who disagree are conspiring against them

23. Has no formal education in the subject they are discussing, shows a lack of understanding of basic concepts in the subject, yet asserts that they know better than people who have worked on the subject in a rigorous way for their entire career

24. Fails to show that they did anything meaningful to check their results for accuracy

25. Claims to know someone else’s thoughts or motivations and uses that to paint a negative picture of them

26. Acknowledges their own bias, but instead of doing something to reduce or eliminate its effects, excuses it by pointing out that they’re not the only one who is biased.