Slide-by-Slide Talking Script

What to say "Good morning. I'm Dicha Zelianivan Arkana, NIM 2241720002. Today I'm presenting my thesis — a preliminary pilot study on applying Kit-Build concept mapping for Japanese reading comprehension among beginner learners."

What to say "Japanese has three writing systems — Hiragana, Katakana, and Kanji. For Indonesian learners whose native language uses Latin script, this is a substantial barrier. Add to that the SOV word order, verb conjugations, and particles — all structurally different from Indonesian — and you get a real comprehension problem. Indonesia has over 700,000 Japanese learners — second most in the world — so motivation is high. But motivation doesn't remove the structural difficulty. The core gap is this: learners can decode individual words, especially with furigana help, but they struggle to connect those words into meaningful relationships. They see the pieces, not the picture."

What to say "Concept mapping is a technique where learners draw nodes for concepts and links for relationships. It externalises understanding. The problem: open-ended maps are impossible to assess automatically — every student draws something different. Kit-Build, developed by Hirashima in 2015, solves this. The teacher creates a goal map — the correct understanding. The system decomposes it into components — nodes and links — forming a 'kit.' Learners reconstruct the map from the kit. Then the system automatically diagnoses matching, missing, and excessive propositions at the link level. The score is the proportion of correctly reconstructed propositions — matching links divided by total goal links, times one hundred percent."

What to say "I investigated three questions. First: what is the within-group reading comprehension score direction after the Kit-Build activity? Note the word 'direction' — not 'improvement' or 'effect.' Second: how do beginner Japanese learners perform on Kit-Build reconstruction tasks? Third: what are learner perceptions and experiences? The scope is deliberately narrow. This is a pilot study with beginner Japanese learners from two Marketing Management classes at Polinema. Reading was assisted — furigana and dictionary were available. The design is one-group pretest-posttest. I want to be very clear upfront: this study makes no causal claims and no claim of platform superiority."

What to say "Here is the full workflow. Step one: the teacher prepares a goal map — the target understanding of the reading passage. Step two: the system decomposes the goal map into a kit — individual nodes and links, like puzzle pieces. Step three: the learner reconstructs the map from those pieces — this is the actual learning activity. Step four: the system compares the learner's map to the goal map and classifies every proposition as matching, missing, or excessive. Because the components are identical, the comparison is exact — no subjective grading. This makes Kit-Build fundamentally different from open-ended concept mapping. It's close-ended, automatic, and gives immediate formative feedback."

What to say "The Kit-Build framework was established by Hirashima in 2015. Alkhateeb showed in 2015 and 2016 that Kit-Build produces better delayed recall than summarisation and selective underlining — important because it suggests the benefit is in retention, not immediate scores. Pailai in 2017 developed analytics for formative assessment. Andoko in 2020 applied it to EFL reading and found it outperformed summarisation. Funaoi in 2011 found that retention gains were specific to kit-covered content. But here's the gap: every Kit-Build reading study has been in English. Japanese adds three scripts, Kanji, and SOV structure. There is no published Kit-Build experiment for Japanese language learning. And no modern web platform integrates the full Kit-Build workflow with a research pipeline."

What to say "The design is a one-group pretest-posttest pilot. I want to be explicit: no control group. That means this study measures direction, not causation. Participants were beginner Japanese learners from two Marketing Management classes at Polinema — convenience sampling, not random. Thirty-five students enrolled, and after data quality checks, 31 were included in analysis. The vast majority reported no JLPT certification, and where JLPT was reported, it was self-reported and unverified. That's important because it means 'beginner' is defined by course enrolment, not standardised test scores."

What to say "Yomilink is a modern web-based Kit-Build platform I developed for this research. It's open-source under the MIT license. The original Kit-Build implementations lacked integrated consent collection, test delivery, data export, and easy web deployment — all of which are necessary for running a study. Yomilink bundles the full Kit-Build core — goal map preparation, kit generation, learner reconstruction, and automatic proposition diagnosis — with a complete research pipeline: consent forms, pre-test and post-test, TAM questionnaire, feedback collection, analytics dashboard, and one-click data export. Everything runs in the browser. No installation required."

What to say "The reading test used 10 multiple-choice questions based on a passage from Minna no Nihongo Chapter 10 — 'Watashi no Uchi,' or 'My House.' Furigana was provided on all Kanji and a dictionary was allowed throughout. The same 10 questions were used for both pre-test and post-test — I'll address that choice in the discussion. The goal map contained 7 propositions covering spatial and functional relationships in the passage. For technology acceptance, I used Davis's TAM model — perceived usefulness and perceived ease of use — on a 5-point Likert scale with a threshold of 3.5. Three open-ended feedback questions were analysed thematically. The study ran over three sessions: Day 1 was an offline introduction and demo, Day 2 was familiarisation and instrument calibration — the results from Day 2 were excluded — and Day 3 was the main online session where everything happened in sequence through Yomilink."

What to say "For descriptive statistics, I report mean, median, standard deviation, and trend counts. For the first research question on score direction, I started with Shapiro-Wilk for normality — the data was non-normal, as expected with bounded scores. I used the paired t-test as the primary test, relying on the central limit theorem since n is above 30. Wilcoxon signed-rank served as the non-parametric confirmation. Cohen's d gives the effect size. I also ran IQR-based outlier sensitivity. For the second question on Kit-Build reconstruction, I used Spearman correlation between Kit-Build score and score change, Pearson correlation between pre-test and gain, and a median split with Welch's t-test and Mann-Whitney, with Hedges' g for effect size. For the third question on perceptions, I compared TAM construct means against the 3.5 threshold and used thematic coding for open feedback. Throughout, the emphasis is: direction, not causation."

What to say "Before the main results, a note on data quality. Thirty-five students participated. I excluded participants whose test completion time was under 60 seconds — suggesting rushed or inattentive responses. This left 31 valid participants for analysis. The mean age was about 20, with roughly a year and a half of Japanese study on average. Prior course scores averaged in the mid-70s out of 100. The vast majority had no JLPT certification. This confirms the sample as genuine beginners with some classroom experience but no standardised proficiency."

What to say "The pre-test mean was 7.06 out of 10. The post-test mean was 7.77 — a gain of 0.71 points. The median stayed at 8. The box plot shows the distributions overlapping substantially, which is expected for a small within-group change. Notice the pre-test distribution is already quite high — the median is 8 out of 10. This is the ceiling effect I'll return to. The standard deviations are large — around 2.4 for both pre and post — which means there was substantial individual variation. Some learners already scored near the top; others had more room to move."

What to say "The dumbbell chart shows every participant's individual trajectory. Each line is one person — the left dot is their pre-test, the right dot is their post-test. Green lines go up, red lines go down. About 65% of participants improved — that's roughly two-thirds. About 19% stayed the same. About 16% declined. The average gain among those who improved was modest. The key observation: the people who improved the most started lower. The people who declined or stayed flat started higher. This pattern — low scorers gain, high scorers don't — is the signature of a ceiling effect."

What to say "The Kit-Build reconstruction scores averaged around 54%. On a goal map of 7 propositions, that's about 3.8 propositions correctly reconstructed on average. The median was similar, but the range is striking — from zero to one hundred percent. This wide spread tells us that some learners fully grasped the passage structure and could reconstruct it independently, while others struggled significantly. The highlighted point in the box is important: the Kit-Build score captures more than just structural understanding. Because the kit components contain Kanji that learners must decode, the score also reflects extraction ability. This makes it a broader construct than the multiple-choice test, which provides recognition cues. I'll return to this in the discussion."

What to say "Engagement dropped from pre-test to post-test — engaged participants fell while marginal and rushed categories rose. This is almost certainly fatigue. The online session had six phases stacked back-to-back — registration, consent, pre-test, Kit-Build activity, post-test, TAM, feedback — all on top of processing Kanji with a novel task. This is extraneous cognitive load. It's not that learners lost interest in the platform. It's that 30 to 40 minutes of sustained Kanji processing is exhausting for beginners. Future studies should split this across multiple sessions."

What to say "All 35 participants completed the TAM questionnaire. Both perceived usefulness and perceived ease of use had means above the 3.5 threshold — indicating positive initial acceptance. The per-item bar chart shows most items clustered above the threshold line. This is early-stage evidence — it doesn't prove long-term adoption — but it tells us that beginners found Yomilink usable and saw value in it on first encounter. For a pilot study testing a novel platform with a novel task, clearing the acceptance threshold is a meaningful result."

What to say "The most common positive theme — mentioned 24 times — was that the platform helped learning. Learners felt the reconstruction activity made them think about how ideas connect. The most common difficulty was initial confusion — 12 participants mentioned the task was unfamiliar at first. This is expected for a novel activity and reinforces the need for familiarisation sessions, which we included as Day 2. Notably, despite furigana being provided, some learners still reported Kanji difficulty. This suggests that furigana alone doesn't eliminate the decoding barrier — it helps with pronunciation but not necessarily with mapping characters to meaning under time pressure. Top suggestions were more vocabulary practice and better user interface — both actionable for future development."

What to say "Shapiro-Wilk confirmed that the data is non-normal — as expected with bounded scores and a ceiling effect. The paired t-test, which is robust at n above 30 via the central limit theorem, showed a statistically significant difference: t of 2.219 with 30 degrees of freedom, p equals 0.034. Cohen's d was 0.399 — a small-to-medium effect. The Wilcoxon signed-rank test, which doesn't assume normality, confirmed this with p equals 0.014. I also ran a sensitivity analysis excluding one outlier — a participant who dropped 6 points. With that participant excluded, the t-value rose to 3.52 with p less than 0.001 and a medium effect size. The result holds: there is a statistically significant positive score direction."

What to say "The Spearman correlation between Kit-Build score and score change was 0.198 — weak and not statistically significant, with p equals 0.284. The scatter plot confirms this — there's no visible pattern. This might seem like a negative result, but it's actually informative. In English Kit-Build studies, reconstruction score and comprehension test score tend to correlate because both measure the same underlying construct: understanding of relationships. In Japanese, the tasks diverge. The multiple-choice test provides answer options that scaffold Kanji recognition — it's a recognition task. The Kit-Build reconstruction requires independent proposition extraction — it's a production task. The KB score captures extraction ability that the MCQ does not. So the weak correlation is not a failure of Kit-Build — it's evidence that orthographic complexity introduces a construct distinction that doesn't exist in alphabetic-language studies."

What to say "The Pearson correlation between pre-test score and gain was negative 0.549 — highly significant at p equals 0.001. This means the higher someone scored at pre-test, the less they gained. The median split at 8 makes this concrete. Nineteen learners were at or below the median — they gained an average of 1.53 points. Twelve learners were above the median — they actually declined slightly, by 0.33 points on average. The difference between these groups is large: Welch's t of 2.7, p equals 0.013, and Hedges' g of 0.988 — that's a large effect. A substantial proportion of participants scored 8 or above at pre-test, and some scored a perfect 10 — leaving literally zero room to improve. The ceiling effect is real and substantial. It doesn't invalidate the positive direction — if anything, it means the true effect might be larger if measured with more difficult material — but it constrains what we can conclude from this specific test."

What to say "To summarise the results. One: Yomilink proved feasible — the full Kit-Build plus research workflow ran online with 35 participants. Two: scores moved in a positive direction, from 7.06 to 7.77 out of 10, statistically significant with a small-to-medium effect. Three: Kit-Build reconstruction averaged around 54%, with wide variation. Four: there was no correlation between reconstruction quality and score gain — I've argued this reflects construct contamination from Kanji, not framework failure. Five: a substantial ceiling effect was confirmed — low-scorers gained, high-scorers did not. Six: engagement dropped, likely due to cognitive load from the single-session design. Seven: technology acceptance was positive, with both PU and PEoU above the threshold."

What to say "This is the most important slide for examiners. What can this study conclude? Yomilink is feasible for online Kit-Build with Japanese readers. The score direction is positive, providing evidence to justify future controlled testing. The pilot procedure is tested and documented. Initial acceptance is positive. What can this study not conclude? Kit-Build causes improvement — no control group. Yomilink outperforms the original Kit-Build — no direct comparison. Results generalise beyond this sample — they don't. Every threat to validity is acknowledged: no control group, same pre and post items, assisted conditions, familiar passage, ceiling effect. These aren't accidents — they're features of a pilot design. The purpose of a pilot is to test feasibility and gather initial data for future controlled studies. This pilot achieved both."

What to say "Both the paired t-test and Wilcoxon agree — the score direction is positive. But there are important caveats. Without a control group, we can't attribute this to Kit-Build. The same items were used twice — but interestingly, the high-scorers declined, which argues against a pure practice effect. If practice alone drove scores up, everyone should have improved. They didn't. The reading was assisted — furigana and dictionary were always available — so this is assisted comprehension, not unaided. And the Day 2 familiarisation session used the same passage, which may have inflated the pre-test baseline. There's also a schema theory angle: the passage was actual course material, confirmed by the lecturer. Many learners already had adequate background knowledge — they understood the house layout before they read about it. This makes the test easier and contributes to the ceiling. The right-hand box raises a deeper point about proficiency: beginners may be trapped in bottom-up processing, over-relying on pop-up lookups. Furigana and a dictionary help with individual words, but they may not be enough scaffolding to bridge from word-level decoding to relational understanding. This is the bottleneck."

What to say "The lack of correlation between Kit-Build score and test gain is one of the most interesting findings. In English Kit-Build studies, reconstruction and multiple-choice tests measure the same thing — do you see how concepts connect? Both are about structural understanding. In Japanese, they diverge. The MCQ provides answer options in Japanese — these scaffold Kanji recognition. It's a recognition task. The Kit-Build reconstruction requires learners to independently extract propositions from Kanji-laden components — it's a production task. The KB score captures extraction ability that the MCQ does not. This means the two instruments measure overlapping but distinct constructs, and the weak correlation follows. This interpretation is consistent with prior Kit-Build work. Alkhateeb found no immediate gain from Kit-Build but better delayed recall. Funaoi found retention gains specific to kit-covered content. Both suggest the payoff may be in long-term retention, which this study did not measure. The qualitative feedback aligns — learners reported Kanji difficulty and requested auto-translation features, confirming that extraction was the bottleneck."

What to say "A methodological strength worth highlighting: the Day 2 calibration session identified three ambiguous distractors and two items with uncovered vocabulary. These were revised into the final 10-item instrument. This is exactly why pilots exist — to refine instruments before the main data collection. For engagement, the drop from pre to post is interpretable as cognitive load — six phases stacked in one session, on top of Kanji processing and a novel task. Splitting the online session across multiple days is the most direct fix. The full limitations list includes: no control group, identical pre and post items, assisted reading conditions, Day 2 carryover inflating baseline, ceiling effect from the familiar passage, a small localised sample, TAM measured at one time point with recency bias, a single reading passage, and no delayed post-test to measure retention. Each of these is addressed in the thesis and each points to a specific improvement for future work."

What to say "To conclude. Yomilink is a modern, open-source web-based Kit-Build platform — demonstrated feasible for Japanese reading comprehension with 35 learners. The score data pointed in a positive direction — 7.06 to 7.77 out of 10, statistically significant, but cautiously interpreted as a single-group pilot with assisted conditions and a ceiling effect. The ceiling effect was quantified: the median split confirmed that low-scorers gained substantially more than high-scorers, with a large Hedges' g. The missing correlation between Kit-Build score and gain is not a framework failure — it reflects construct divergence driven by Japanese orthography and was underpowered. The calibration session refined the instrument, and initial technology acceptance was positive. This study makes three contributions. One: Yomilink itself — a modern, open-source platform. Two: early evidence that Kit-Build can be applied to Japanese reading, with documented caveats. Three: a pilot-tested procedure that future controlled studies can build on."

What to say "For future research, I recommend a larger and more diverse sample across proficiency levels and study programs. Parallel test forms should replace identical pre and post items. A delayed post-test is essential — both Alkhateeb and Funaoi's work suggests the Kit-Build payoff may be in retention, not immediate scores. Multiple passages would control for passage-specific effects. For practice, Kit-Build makes sense as a supplementary formative activity — especially in non-language programs where learners need to read Japanese but aren't linguistics majors. The task goal should be clarified: assisted reading with dictionary and text access is a valid construct, but it should be named as such. Learners need familiarisation time before assessment — the confusion reported in feedback reinforces this. And the platform itself should receive clearer tutorials, better interaction design, and richer analytics. A direct comparison between Yomilink and the original Kit-Build platform would be needed to make any claim of improvement."

What to say "Thank you. I'm happy to take your questions."