All of us have at some point in life have taken standardized tests like SATs, GMATs, GREs, and TOEFL. A test of progressive difficulty that aims to assess our verbal, quantitative, and pattern-seeking ability. We have all taken surveys, Likert scale items, multiple response questions, and MCQs but how do we know that it measures what it does? What does that actually measure?

That is where psychometrics comes in. It is the field of study that helps researchers develop and evaluate through complex mathematical modeling tools, the reliability, and validity of our instruments. With the power of machine learning and large datasets, it has become easier to quantify social phenomena like political opinions, educational trends, criminal behavior, and personality preferences.

On Tuesday, February 16th I will be talking to one of the leading psychometricians Dr. Okan Bulut. He is a professor of Educational Psychology at the University of Alberta and has worked at premier institutions developing assessments at Pearson, PluralSight, and the American Institute of Research. He is the author of the book ‘Handbook of educational measurement and psychometrics using R.’ You are in for a real treat!

Video Link:

Audio Link: 

Timestamps of the Conversation:

00:00 intro

01:19 Canadian Wilderness

07:55 Early Education in Ayden, Turkey

12:00 Studying for Happiness vs Peer Pressure and Standardized Tests

21:22 Study Culture and Ethics East vs West. Fun vs Perfectionism

33:05 Parental Support and Achievement Orientation

46:00 Turkish Blockbuster series Ertugrul, Ishq Memnun and Letting out Steam

49:05 Work at Pearson with Psychometric Evaluations, Bell Curve by Charles Murray and Fairness

54:35 Towards Data Science Articles on Psychometrics and book ‘Educational Measurements using R’

01:00:21 Construction of Items in Surveys and Phrasing. Fairness in Quantitative Methods

01:08:00 Clark University’s Ph.D. Student expelled for reporting P-hacking by Supervisor, Academic Dishonesty

01:15:10 Peter Boghossian’s hoax papers in Feminist Journals

01:18:40 Politics in Academia and Left-wing Dominance of Campuses

01:22:40 Recommendation System for Course Selection and Editor’s Personal Biases and Ignorance

01:2:08 Machine vs Human Predicted Algorithms and Big Data

01:32:55 Piaget, Vygotsky and Edith Ackerman

01:33:10 Homeschooling, Popularity in Alberta, Public School System

01:38:16 College Degrees and Recruitment. Robin Sharma, Buddhism, Rat Race

01:45:42 Dilemma of Choice, Nature vs Nurture Debate, Blank Slate, Behaviourism

01:49:08 Nomothetic vs Idiographic Approach. Tests like Rorschach and Validity and Turkish Voodoo

01:52:02 Big Five Test, MMPI, and ESP

01:55:15 Machine Learning Algorithms and Behavioral Prediction through Facebook Likes

02:00:46 Standardized Tests and Admissions in Ivy Leagues

02:03:08 Test Design and Administration, Disregard for Introversion/Extraversion Preferences

02:06:55 Women leaving STEM fields and Violence on Campuses

02:12:50 Automatic Generation of Comprehension Passages, GPT3, and Transformers. OpenAI’s restrictive Access

02:17:14 One Thing You Want to Change

Get 30% off of his book with this code ASA21…

Okan’s Profile:

Personal Website:

Okan’s Blog:

Full Episodes Playlist link:

Clips Playlist link:

Apple Podcast Link:



Minhaaj Rehman is CEO & Chief Data Scientist of Psyda Solutions, an AI-enabled academic and industrial research agency focused on psychographic profiling and value generation through machine learning and deep learning.


✩ Website –

✩ Minhaaj Podcast –

✩ Twitter –

✩ Facebook –

✩ ResearchGate –

✩ Linkedin –

✩ Buy Me a Coffee (I love it!) –

Episode Sponsors

Show Notes


Related Episodes