1. The Education Gradebook

Inside the metrics machine: USF uses big data to find, and help, overburdened students

Students gather for a December 2016 commencement ceremony in the Sun Dome at the University of South Florida, where officials are mining data to spot patterns of success and pinpoint students veering off track so they can get the support they need. Already, the effort has lifted USF's graduation and retention rates to new highs.
Students gather for a December 2016 commencement ceremony in the Sun Dome at the University of South Florida, where officials are mining data to spot patterns of success and pinpoint students veering off track so they can get the support they need. Already, the effort has lifted USF's graduation and retention rates to new highs.
Published Apr. 2, 2017

TAMPA — Paul Dosal left Hillsborough High School with an enviable GPA, seventh in a class of 700, destined for college success.

"And I ran straight into the buzz saw of English comp 1," Dosal said. Each C chipped away at his conviction.

"Paul is really struggling," his mother told a family friend. That man sat Dosal down in his church office and, in a life-altering 30 minutes, showed him how to write.

"I didn't go see the instructor during office hours. I didn't go to the tutoring center," Dosal said. "Someone had to pull me up."

Left to chance, Dosal could have been one of those talented students who slip through the cracks. With a C, he would have passed. But some classes are so foundational, a growing pool of research shows, that one mediocre grade can spell serious trouble in the semesters to come.

Dosal is now a vice president at the University of South Florida, searching for students like himself, who might not even realize they're struggling. He wants to take chance out of the equation.

Enter big data. Powerful software maps patterns of success, then pinpoints students veering off track so they can get the support they need. Already, the effort has lifted USF's graduation and retention rates to new highs. And with the graduation rate, at least, the achievement gap between white and black students has been erased.

At the same time, fascinating patterns have begun to reveal themselves, shaping the way USF approaches its students. Demography is not destiny, as USF leaders like to say. Sometimes all it takes is a nudge in the right direction.

Long-held beliefs about student behavior are falling away across the nation.

Take this: A bad grade in Conceptual Foundations of Nursing used to be the signal that a Georgia State nursing student wouldn't graduate. But decades of data revealed the true turning point was introductory math. No matter how students performed in nursing, it was math that made the difference. Perform poorly, and chances of graduating plummet.

Universities have long collected student data, usually for institutional reports. Insights came too late to help the students who, by then, were gone.

But in the last six or seven years, data companies have begun offering real-time windows into student decisions while there's still time to intervene. Though predictive analytics have been used in health care and sports for years, higher education is just beginning to grasp the potential.

Universities can see now that 44 percent of students they lose have a 3.0 GPA or higher, thanks to Civitas Learning software. They can track engagement online, where students write on discussion boards and turn in assignments. If the software spots last-minute cramming, a sign of trouble, it may flag a student's name so advisers can check in.

Crunching the data, leaders can spot troublesome courses that need an overhaul, or a demand for support services.

Follow what’s happening in Tampa Bay schools

Follow what’s happening in Tampa Bay schools

Subscribe to our free Gradebook newsletter

We’ll break down the local and state education developments you need to know every Thursday.

You’re all signed up!

Want more of our free, weekly newsletters in your inbox? Let’s get started.

Explore all your options

At USF, for instance, female students are power users of the career and counseling centers. Men are more reluctant, and their graduation rates lag far behind. Now USF is plotting ways to bridge the gap.

"We're trying to move from a passive approach where we all wait in our offices to where we go out there," said Dosal, vice president of student affairs and student success.

Through a contract with Civitas, USF will be able to see clearly which of its courses are strong graduation predictors. Then it can pour extra resources into those essential subjects.

"There's a sense here that one of the key predictors of success in the College of Engineering is success in English comp," said associate professor Tom Miller. "I guess engineers need to be able to write."

So far, he said, USF has only seen the tip of the data iceberg.

• • •

In 2001, when Miller arrived at USF, "student success" hadn't yet become USF's driving force. Officials focused on getting students in the door, but barely a fifth of them graduated in four years.

"Jeez, we ought to be able to do better than that," Miller thought. He implemented a survey for incoming freshmen to gauge the extra support they might need. Students who expected a lot of classroom attention might be led to peer tutoring. Working students might get extra advising so they could stay on track.

More students began sticking around, then graduating. USF added advisers and built living-learning communities to boost a community spirit.

Around 2012, though, retention rates plateaued at about 86 percent, short of USF's goal. Two years later, Florida lawmakers began hitching university funding to performance in key metrics, including retention, increasing the pressure to raise the bar.

Civitas promised game-changing insights for $286,000 per year. USF signed up, then formed a Persistence Committee with Miller as chairman, connecting students to advocates from advising to financial aid to resident life and beyond.

Once again, the rates for retaining and graduating students began to rise, putting USF within inches of the state's highest standards and getting it even closer to millions in bonus funding that would help hire a new wave of faculty.

• • •

This is how predictive analytics work at a university:

Maybe you're an engineering student. Your GPA is solid, but it has been hard to focus lately. Your dad's on sick leave, and now you're not sure you'll make rent. Your assignments are increasingly late. Sometimes you skip posting on the discussion board.

Behind the scenes, an algorithm sees that you're slipping behind your peers. Your name gets flagged. USF checks out your lackluster engagement, then dispatches someone to reach out, like your resident assistant or a professor.

All you know is that someone familiar knocks on your door and says, "Hey, let's talk about how you're doing in class. I want you to succeed."

You open up about the money troubles. (Or maybe it's depression, or confusion about class registration.) Your professor tells you about USF's emergency fund, or the counseling center, or the advisers waiting to help.

Your sprawling university feels a little smaller.

• • •

The field of predictive analytics has developed more quickly than an ethical framework around it, and some experts advise caution.

Institutions must be careful not to breach student privacy or to limit students' options for their own gain.

"There's a risk of the prediction becoming a self-fulfilling prophecy," said Martin Kurzweil, a program director at Ithaka S + R, an education research organization.

Say a pre-med student gets a poor grade in algebra, which signals that he will likely struggle in his chosen major. A tactless adviser risks scaring the student away from his dream if she reveals the prediction.

Another issue would be an institution deliberately pushing students like him into degrees they don't want, just to increase graduation statistics, or worse, weeding them out altogether.

Critics point to Mount St. Mary's University, where the previous president once said he wanted to "drown the bunnies" who struggled.

"It's incumbent on an administration that has access to the information to take the steps to help that student," Kurzweil said.

Mark David Milliron, the founder of Civitas Learning, calls this "the moral imperative of knowing."

"Now you reach out and move the needle," he said. "Set up a human connection."

Advocates at USF use motivational interviewing and don't let students know they've been deemed at-risk. They can help develop a Plan B "without crushing their dreams," Miller said.

• • •

How big data will shape the classroom remains unclear.

USF has not yet signed up for faculty software, which would let professors dive into the ways students are engaging in material online.

"We don't want those biases creeping into the teaching and grading process," Dosal said.

But he said faculty play a role in spotting issues the data might miss, like overburdened students who go to class but fall asleep in the back row.

USF faculty have welcomed the push for deeper data, said James Garey, chairman of the cell and molecular biology department.

"When you have 40,000 students, you can't look at every one of them," he said. "We're not going to drop the bar intellectually, but we want to do whatever we can to make them do better."

A few other Florida schools have hired Civitas, including the University of Central Florida and St. Petersburg College.

Civitas has helped St. Petersburg College connect some dots. Students who go to tutoring have a higher chance of graduating, but tutoring was long portrayed as punitive.

"We've really turned that around," said Jesse Coraggio, vice president of institutional effectiveness and academic services.

The college turned musty corners of the library into bright, Barnes & Noble-esque spaces where students can chat with teachers. And data showed that two courses lay essential groundwork for future studies, prompting a policy change. Students who take intermediate algebra and composition I now must do so early in their career.

USF St. Petersburg, where the six-year graduation rate is just 38 percent, will roll out the Civitas system soon. Martin Tadlock, regional vice chancellor for academic affairs, looks forward to early alerts for struggling students. But the school's plan to build a home for its students, he said, will need to go far beyond "a red flag on a piece of software."

Two-thirds of students commute, and a significant number transfer elsewhere. The school lacks athletics, plentiful housing and a full residential experience.

"We just don't have that ability to ground them as much as we'd like," Tadlock said. "You don't want to leave your family, so we've got to create that sense of family here."

• • •

"You might have noticed that this university is focused on performance metrics," Dosal said from the podium in late March.

Laughter rang out across the USF ballroom, where nearly 300 staff and faculty sat at tables covered in glossy green tablecloths to celebrate student success initiatives. They broke into applause when Dosal talked about USF's rising six-year graduation rate, now at 67 percent.

Milliron, the Civitas founder, took the microphone. He was the first in his family to go to college, he said. But now he has kids who wore University of Texas onesies. They've got a support system to help them tackle college.

But there are plenty of kids like him at USF now, he said. First-generation students who don't know what all of the abbreviations mean. Others who need help navigating the bureaucracy. Sometimes they stay hidden until it's too late.

These new tools are powerful, he said. He hopes they can give everyone a chance.

Contact or (727) 893-8321.


This site no longer supports your current browser. Please use a modern and up-to-date browser version for the best experience.

Chrome Firefox Safari Edge