Numbers Survival Guide

Every statistic in your slides — what it means, how to explain it, what examiners will ask.

1   Your Sample

Enrolled
35
All students in two Marketing Management classes at Polinema who signed up.
Analysed (valid)
31
After removing 4 who rushed through the test (under 60 seconds each). These are the only ones in the main analysis.
Excluded
4
2 had both pre and post invalid. 2 had only post invalid. All completed the test in under 60 seconds — impossibly fast for reading Japanese text and answering 10 MCQs.
If asked "why 60 seconds?" — say it's a data quality threshold. A 10-item Japanese reading MCQ cannot be genuinely completed in under a minute. It's a standard way to filter out random clicking.
Mean age
19.97 yr
Typical for a first/second-year university course in Indonesia.
No JLPT certification
85.7%
30 out of 35. These are genuine beginners — no formal Japanese proficiency credential. JLPT was self-reported, not verified.
Mean study duration
17.71 mo
About 1.5 semesters of Japanese study. Enough to know hiragana/katakana, but still very early for kanji.
Mean prior score
77.38 / 100
Self-reported prior Japanese knowledge (separate from the pre-test). Quite high — this class may have stronger-than-average beginners.
Why only 35 participants? Isn't that too small?
This is a pilot study. The goal was to test feasibility — does the platform work, does the procedure run smoothly, can we collect clean data? For that purpose, 31 valid responses is enough. For the paired t-test, the central limit theorem kicks in at n ≥ 30, so we just barely meet that threshold. For a controlled experiment, you'd want 60+.
Pilot = "let's see if this works before we go big." 31 is fine for that.

2   Pre-Test vs Post-Test Scores

MeanMedianSD
Pre-test7.138.002.35
Post-test7.949.002.10
Difference (Δ)+0.811.002.02
Scores out of 10. On average, students went from 7.1 → 7.9. A gain of less than 1 point on average.
A gain of only 0.81 — that seems tiny. Is it even worth noting?
Two things. First, the pre-test was already high (7.13/10). When most students start at 7 or 8, there's very little room to go up. Second, the question isn't "how big is the gain" but "which direction did it go." The gain is positive and statistically significant (p = 0.017). For a pilot with no control group, establishing a positive direction is the right goal. A controlled experiment would then measure the actual magnitude.
What does standard deviation (SD) mean here?
SD measures how spread out the scores are. Pre-test SD = 2.35 means most scores fell roughly between 4.8 and 9.4. Post-test SD = 2.10 is slightly tighter — scores clustered a bit more after the intervention. The delta SD of 2.02 tells you there's substantial individual variation in how much people changed — some improved a lot, some stayed flat, some declined.
SD = "how much do people differ from the average?" Bigger SD = more variety in scores.
Why report both mean and median?
Because the data is skewed by the ceiling effect. The mean (7.13) is pulled down by the few low scorers. The median (8.00) is the middle value — half scored above 8, half below. When mean and median differ a lot, it tells you the distribution isn't symmetrical. Here they're close enough that both are useful: mean for statistical tests, median for the "typical" student.
Mean = average. Median = middle score. When they're different, the data is lopsided.

3   Individual Changes

Improved
16 (51.6%)
Score went up after KB activity.
Stayed the same
9 (29.0%)
Score didn't change.
Declined
6 (19.4%)
Score went down.
Avg gain (improvers only)
+2.25 pts
Among the 16 who improved, the average improvement was 2.25 out of 10.
6 people declined — doesn't that mean KB hurt them?
No. Without a control group, you can't say KB caused anything. The 6 who declined are within normal test-retest variation — people have good days and bad days, pay attention differently, or guess differently. The fact that the overall trend is positive (16 improved vs 6 declined) matters more than any individual case. The high-pre group decline is especially explainable: they scored near the ceiling at pre-test, so any random variation is more likely to push them down.
Never say "KB made them worse." Say "some scores declined, which is normal variation."
Why is the avg gain (2.25) so different from the overall mean delta (0.81)?
Because the 0.81 includes everyone — the 16 who improved, the 9 who stayed the same, and the 6 who declined. The 2.25 only counts the 16 improvers. It's like: "if you improved, by how much?" vs "on average, did everyone improve?" Both are valid, they answer different questions.
0.81 = average for ALL 31 people. 2.25 = average for only the 16 who improved.

4   KB Reconstruction Score

Mean KB score
60.39%
Out of 7 propositions, learners reconstructed ~4 correctly on average.
Range
0% – 100%
Some got everything wrong, some got everything right. Huge spread.
Median KB score
71%
Half the class scored above 71%, half below. Higher than the mean, meaning a few low scores pulled the average down.
Props reconstructed
4 / 7
The goal map had 7 propositions (relationships). Average learner got about 4 of them right.
60% seems low. Does that mean learners struggled with KB?
It means the task was challenging — which is actually expected and even desirable. If everyone scored 95%, the task wouldn't be differentiating. The wide spread (0% to 100%) shows it captures real variation in structural understanding. Remember, this is a beginner group doing concept mapping in a foreign language for the first time. 60% is a reasonable starting point.
What does "wide spread" mean and why does it matter?
The range is 0% to 100% — the full possible range. This means the KB task isn't just a "everyone passes" or "everyone fails" instrument. It's actually measuring differences between learners. That's good for a research tool: if every score was the same, you couldn't study what factors predict KB performance.
Wide spread = the test can tell the difference between learners. That's a feature, not a bug.

5   The Statistical Tests (RQ1)

Shapiro–Wilk (normality test)

Shapiro–Wilk tests whether data follows a bell curve (normal distribution). Results: Pre: p = 0.019, Post: p < 0.001, Delta: p = 0.004.
All three are below 0.05, meaning the data is not normal.
Not normal = scores aren't spread in a neat bell curve. They're clumped near the top (ceiling effect). This is expected and doesn't ruin the analysis.
Why non-normal? Bounded scores (0–10) with a ceiling effect naturally skew left. This is a property of the measurement, not a flaw.

Paired t-test (the main test)

t-value
2.219
The t-value measures how far apart the pre and post means are relative to the variation. Bigger = more difference. 2.219 is modest but enough.
p-value
0.017
The probability of seeing this difference (or bigger) by pure chance if there were actually no effect. 0.017 = 1.7% chance. Below 0.05 = "statistically significant."
Why use a paired t-test if the data isn't normal?
Because of the Central Limit Theorem (CLT). At n ≥ 30, the sampling distribution of the mean is approximately normal even if the underlying data isn't. Our n = 31 just clears this threshold. This is standard practice — most statisticians agree the t-test is robust to moderate non-normality at this sample size. We also report Wilcoxon (the non-parametric version) as a supplement, and it agrees (p = 0.014).
Rule of thumb: t-test is okay if n ≥ 30, even with non-normal data. We have 31. We also double-check with Wilcoxon.
What does p = 0.017 actually mean in plain English?
If there were truly no difference between pre and post scores — if the KB activity did nothing — there would be only a 1.7% chance of seeing a gain this large or larger just from random luck. Since 1.7% is very low (below the standard 5% threshold), we conclude the positive direction is unlikely to be pure chance.
Do NOT say "there's a 98.3% chance KB worked." That's a common misinterpretation of p-values. P-value = "probability of this data if null hypothesis is true," NOT "probability the alternative is true."

Cohen's d (effect size)

d = 0.399
Effect size measures how big the difference is, regardless of sample size. Cohen's conventions: d ≈ 0.2 = small, d ≈ 0.5 = medium, d ≈ 0.8 = large.
Our d = 0.399 = small-to-medium.
p-value asks "is there a difference?" Effect size asks "how big is it?" A result can be statistically significant but still small in practice.
So the effect is small — isn't that a problem?
For a pilot study, no. Small effects are expected when: (1) the intervention is a single session, (2) there's no control group so the true KB effect is diluted by noise, (3) the ceiling effect compresses the score range, and (4) the sample is small. The point of the pilot is to establish direction and feasibility, not to demonstrate a large effect. A controlled experiment with a larger sample would give a more accurate effect size estimate.

Wilcoxon signed-rank (non-parametric backup)

W statistic
52.000
The Wilcoxon W is the sum of signed ranks. You don't need to explain the formula — just know it's the non-parametric version of the paired t-test.
p-value
0.014
Agrees with the t-test (0.017). Both say: positive direction, statistically significant.
Why report both t-test and Wilcoxon?
They answer the same question with different assumptions. The t-test assumes (approximately) normal data; Wilcoxon makes no distributional assumption. When both agree, the result is more trustworthy — it's not an artifact of which test you chose. Think of it as a double-check.
Two different tests, same answer = more confidence. Like checking your work with two different methods.

Outlier analysis

Outlier detected
1
One participant had Δ = −6 (score dropped 6 points). IQR fence: [−3, 5]. This point fell outside.
Without outlier
t = 3.520, p < 0.001
Result gets even stronger. The finding isn't driven by one extreme case.
What is IQR and why use it for outliers?
IQR = Interquartile Range = the range of the middle 50% of scores. The "fences" are calculated as Q1 − 1.5×IQR and Q3 + 1.5×IQR. Anything outside those fences is flagged as a potential outlier. It's a standard, rule-based method — no subjective judgment needed. Here, one person had Δ = −6, well below the lower fence of −3.
IQR method = a mathematical rule for "what counts as unusual." One person's drop was unusually large.
Why keep the outlier in the main analysis?
Because excluding data points just because they're extreme is dangerous — you can manufacture any result by cherry-picking. The standard approach: report the main analysis with everyone, then show a sensitivity analysis excluding the outlier. If both agree (they do here), the finding is robust. If they disagree, you'd need to investigate why.
Never exclude an outlier just because it's inconvenient. Always show both analyses.

6   KB Score vs Reading Gain (RQ2)

Spearman ρ
0.198
Weak positive correlation between KB score and reading gain. Essentially no relationship.
p-value
0.284
Well above 0.05. Not statistically significant. We cannot reject the idea that there's zero correlation.
No correlation — does that mean KB is useless?
No. It means KB score and MCQ reading gain measure different things. In Japanese: MCQ answer options help you recognize kanji (recognition), while KB requires you to independently extract propositions from the passage (production). So KB score = structure + decoding ability, while MCQ = recognition + comprehension. They're related but not the same construct. Also, n = 31 is underpowered for correlation — you'd need 60+ to detect a correlation of this magnitude reliably.
No correlation ≠ KB doesn't work. It means the KB score and the MCQ test measure different skills.
Why use Spearman instead of Pearson here?
Spearman ρ works on ranks, not raw values. It's more robust to non-normal data and outliers. Since both KB scores and delta scores are skewed (ceiling effect, wide spread), Spearman is the safer choice. It asks "do people who rank high on KB also rank high on gain?" rather than assuming a linear relationship between the raw numbers.
Spearman = "does the ORDER match?" Pearson = "does the LINEAR RELATIONSHIP match?" Spearman is more forgiving with messy data.

7   Ceiling Effect (Median Split)

Pearson correlation: pre-score vs gain

r
−0.549
Moderate-to-strong negative correlation. Higher pre-score = less gain (or more decline).
p
0.001
Highly significant. This isn't random.
r = −0.549 means: the better you did at the start, the less you improved. This is the textbook signature of a ceiling effect.

Median split groups

GroupnMean preMean ΔSD Δ
Low pre (≤ 8)195.68+1.531.78
High pre (> 8)129.42−0.331.92
Welch t
2.701
Tests whether the two groups' gains are different. Significant.
Hedges' g
0.988
Large effect. The gap between low-pre and high-pre gains is substantial.
Why Hedges' g and not Cohen's d?
Hedges' g is a corrected version of Cohen's d for small samples. Cohen's d slightly overestimates effect size when n is small. Hedges' g adds a correction factor that makes it more accurate for the group sizes here (19 and 12). At larger samples (n > 50), they're virtually identical. For n = 19 and 12, g is the more honest number.
Hedges' g = Cohen's d but corrected for small samples. More accurate when n is under ~50.
g = 0.988 is large — but the high-pre group declined. What does that mean?
It means there's a big gap between how the two groups changed. Low-pre gained +1.53, high-pre declined −0.33. The "large" effect size describes this gap, not whether KB worked. The high-pre decline is likely test-retest noise at the ceiling — when you're at 9.4/10, any random variation pushes you down, not up. This is why we say "ceiling effect confirmed," not "KB hurts advanced learners."
The high-pre decline is a ceiling artefact, not evidence of harm. Never say otherwise.
What's the difference between the two p-values — Welch t (0.013) and Mann-Whitney (0.027)?
Same story as paired t vs Wilcoxon. Welch t assumes (approximately) normal data within each group. Mann-Whitney makes no distributional assumption. Both are below 0.05, both agree. The high-pre group's data is especially non-normal (p < 0.001 on Shapiro-Wilk), so Mann-Whitney is particularly valuable here as a sanity check.

At ceiling and perfect scores

Scored ≥ 8 (at ceiling)
51.6%
More than half the class already scored 8+ at the pre-test. Very little room to improve.
Perfect score (10/10)
16.1%
5 students got a perfect pre-test. Literally impossible to improve.
How do you explain the ceiling effect to an examiner who keeps pressing?
"The passage was course material — many students had already studied it. So at pre-test, 51.6% scored 8 or above, and 16.1% got a perfect 10. When you start near the top, the test can't measure improvement — it can only measure decline from random noise. This compressed our effect size downward. The median split confirms: low-pre students gained +1.53, while high-pre students declined −0.33. The test was too easy for half the class. Future studies should use harder passages or a more discriminating test format."

8   Engagement

Engaged: pre → post
77.1% → 54.3%
Significant drop. More students were "marginal" or "rushed" in the post-test.
Marginal + rushed: pre → post
5.7% → 20.0%
Went from 2 participants to 7. Quadrupled.
Why did engagement drop?
Cognitive fatigue, not disinterest. Participants completed 6 consecutive phases (~30–40 minutes) on top of reading Japanese kanji and performing the novel KB task. That's a lot of sustained attention. The drop is consistent with extraneous cognitive load theory — the task stacking overwhelmed working memory, not the content. A fix would be to split phases across days or add breaks.
They weren't bored. They were exhausted. 6 phases + kanji + new tool = too much at once.

9   Technology Acceptance (RQ3)

Perceived Usefulness (PU)
3.67 / 5
Participants generally agreed the tool is useful for learning. Above the 3.5 threshold.
Perceived Ease of Use (PEoU)
3.55 / 5
Participants generally found it easy to use. Just above 3.5. Lower than PU — some found it confusing at first.
PEoU is only slightly above 3.5 — isn't that barely passing?
It's above the threshold, but yes, it's the weaker construct. This matches the qualitative feedback: "initial confusion" was the top difficulty theme (12 responses). PEoU would improve with a better onboarding tutorial, clearer instructions, and more familiarisation time before the actual assessment. The Day 1 demo and Day 2 calibration were designed to address this, but one session may not be enough for a completely novel tool.
What does the 3.5 threshold mean exactly?
On a 5-point Likert scale (1 = Strongly Disagree, 5 = Strongly Agree), 3.5 is the midpoint between "Neutral" (3) and "Agree" (4). It's a widely used cutoff in TAM research — scores above it indicate the technology has moved past neutrality into positive acceptance territory. It doesn't mean "excellent" — it means "lean toward agree, not neutral."
3.5 on a 5-point scale = "I slightly agree it's useful/easy." Not glowing, but positive.

10   Feedback Themes

ThemeCountWhat it means
Helps learning24Top positive theme — participants felt KB helped them understand the text.
More vocabulary14Most common suggestion — they want more content, not less.
Initial confusion12Top difficulty — didn't understand the KB task at first.
No difficulty9Nearly a third had no issues at all.
Better UI6Interface could be more intuitive.
Kanji difficulty4Even with furigana, kanji was hard.
Audio/pronunciation4Wanted audio support.
24 out of 35 said "helps learning" — isn't that strong?
It's the most frequent positive theme and a good sign for a first exposure. But it's self-reported and collected right after the task — there may be demand characteristics (participants say what they think you want to hear). Combined with the TAM scores, it supports "positive initial acceptance." Actual learning effectiveness needs the controlled experiment.

11   Quick Reference Card

NumberWhat it isWhat to say
35 / 31N / NValid"35 enrolled, 4 excluded for rushing, 31 analysed."
85.7%No JLPT"Genuine beginners — most had no formal certification."
7.13 → 7.94Pre → Post mean"Average score went from 7.1 to 7.9 out of 10."
+0.81Mean delta"Average gain was just under 1 point."
16 / 9 / 6Improved / Same / Declined"Half improved, a third stayed flat, a sixth declined."
60.39%Mean KB score"About 4 out of 7 propositions reconstructed."
p = 0.017Paired t-test"Statistically significant positive direction."
d = 0.399Effect size (paired)"Small-to-medium effect."
p = 0.014Wilcoxon"Non-parametric test confirms the same result."
ρ = 0.198, p = 0.284KB vs Gain"No correlation — they measure different things."
r = −0.549Pre vs Gain"Ceiling effect — high starters can't improve much."
g = 0.988Median split gap"Large difference between low-pre and high-pre groups."
77.1% → 54.3%Engagement"Dropped — likely cognitive fatigue."
3.67 / 3.55PU / PEoU"Both above 3.5 — positive acceptance."