Study overview
Melanoma Genetics Workshop
Educational impact evaluation of a professional development workshop for dermatologists, using a paired pre-post design across 12 clinical skills, knowledge transformation, and implementation barriers.
My contribution
This analysis evaluates a melanoma genetics testing workshop for 47dermatologists using a paired pre-post design. I conducted all statistical analyses: paired t-tests, McNemar tests, Cohen's d effect sizes, and thematic coding of qualitative data.
The study assesses three domains: (1) clinical skill development across 12 competencies, (2) knowledge transformation regarding insurance policy changes, and (3) implementation barriers using the COM-B framework. This page presents the statistical work in an interactive format.
Study status
This study remains unpublished. The content here reflects the statistical contribution for portfolio purposes. A conference poster and abstract were presented at the ASDR Annual Scientific Meeting (July 2024).
Loading workshop data…
The workshop succeeded in building capability and motivation, but 95% of implementation barriers turned out to be systemic — time, cost, access — rather than educational. This is the gap between training and practice change.
Methodology
How we measured impact.
A mixed-method workflow that links statistical change detection with implementation-focused qualitative interpretation. Quantitative and qualitative findings were integrated so improvements in confidence are read alongside real-world constraints.
Analysis strategy
Tap a card to flip
Study design
Three measurement domains, two analytical samples.
Matched pre- and post-workshop surveys administered across two sessions. The analysis combines a full-sample descriptive view with a paired-sample inferential view.
Workshop sessions
Combined data from two workshop administrations: Workshop 1 (2023) and Workshop 2 (July 2024, Perth, Australia). 56 unique participants; response rate 85.7%.
Sample strategy
ALL sample · descriptive
Descriptive statistics
Uses every submitted response to summarize overall baseline and post-workshop patterns. Frequency distributions, means, standard deviations for population-level reporting.
PAIRED sample · inferential
Paired comparisons
Uses matched pre/post respondents to estimate within-person change attributable to the workshop. Paired t-tests, McNemar's tests, Cohen's d for effect size.
Survey structure
Tap a card to flip
Each domain below maps to a specific part of the results pipeline, from baseline attitudes to implementation barriers.
Baseline
Where dermatologists stood before the workshop.
Pre-survey results — who walked in, and what they had already done in practice.
Pre-workshop context
Before the workshop, participants completed a survey capturing their prior experience with melanoma genomic testing. This section presents that baseline as descriptive statistics over the full pre-workshop sample (not the paired inferential subset).
Q3 — Prior clinical experience
In the last 12 months, have you performed any of the following tasks?
Loading baseline…
Q5 — Testing frequency
How often do you offer melanoma genomic testing to appropriate patients?
Loading…
Q6 — Patient conversations
Have you discussed melanoma genomic testing with a patient in the last 12 months?
Loading…
Knowledge shift
Before and after the workshop.
Paired pre/post survey responses on the relevance of melanoma genomic testing (Q1, 5-point Likert) and binary statements about value, insurance, moratorium and stigma (Q2).
Q1 — Relevance of melanoma genomic testing
5-point Likert · Strongly Disagree → Strongly Agree
Loading…
Q2 — Value, insurance, policy & stigma perceptions
% agreement · paired pre/post
Loading…
Skills
All twelve skills improved.
Self-rated confidence on twelve clinical genomic competencies, measured pre and post workshop on a 5-point Likert scale. Each row shows the mean shift from before to after.
Replication finding
12 of 12 skills improved significantly (paired t-test, Benjamini-Hochberg corrected). Mean improvement of +1.34 points on a 1–5 scale; mean Cohen's d of 1.23 (large effect). Sample of N = 45 paired participants.
Pre → post · twelve skills
Hover any row for full statistics
Loading skills…
Skills improved
12/12
Paired t-test, BH-corrected.
Mean improvement
+1.34
On a 1–5 confidence scale.
Mean Cohen's d
1.23
Large effect size (d ≥ 0.8).
Significance
p < 0.001
All 12 skills, after correction.
Satisfaction
Loved the workshop. Lower confidence to implement.
Post-workshop evaluation (Q4–Q9), rated on a 1–10 scale. Activity quality and overall satisfaction sit near the ceiling; confidence to actually offer testing in clinical practice sits noticeably lower.
Key finding
Workshop activities rated 9.02/10 on average, with 86% of participants giving a rating of 8 or higher. But confidence to actually offer genetic testing came in at 7.29/10 — a 1.7-point gap between feeling the workshop was excellent and feeling ready to change practice the next day.
Component ratings (Q4–Q9)
1 = low · 10 = high
Loading…
Overall satisfaction
9.0/10
86% rated 8 or higher.
Confidence to offer
7.3/10
Self-reported readiness to use testing in practice.
Workplace support
7.3/10
Perceived organisational backing for implementation.
Sat ↔ skill r
0.50
Significant positive correlation.
Barriers
Systemic, not educational.
Open-text responses on what's stopping melanoma genomic testing from happening routinely — and what would help. Coded with the COM-B framework (Capability · Opportunity · Motivation).
COM-B verdict
The workshop built capability (skills, d = 1.23) and motivation (satisfaction 9/10). But 95% of named barriers were opportunity-side — systemic constraints around time, funding, access. More training won't close the gap; the healthcare system has to.
Q10 — Barriers to implementation
Open-text · n = 42 responses · thematic coding
Loading…
Q11 — What would help
Open-text · n = 30 responses · thematic coding
Loading…
COM-B framework verdict
Built
C
Capability
The workshop produced significant skill gains across all 12 competencies (mean d = 1.23). Participants left more able to perform the clinical tasks.
Blocked
O
Opportunity
95% of named barriers were systemic: time constraints (57%), cost / funding (26%), access to specialists. The system doesn't yet offer the conditions to apply the skills.
Built
M
Motivation
Workshop satisfaction averaged 9.02/10 and 85.7% rated it 8 or higher. The intent to use testing is there.
Study conclusion
A single educational workshop produced statistically large, uniformly replicated improvements across twelve clinical skills and shifted knowledge dramatically on factual matters like the 2019 moratorium. But participants left more able and willing to use genomic testing while still facing systemic barriers — time, cost, access — that no further training will dissolve. The takeaway is for programme design: effective workshops are necessary but not sufficient for practice change. The next intervention should be operational, not educational.