That's according to a report released Thursday from credit rating agency Moody's Investors Service. While higher college enrollment rates and rising tuition costs used to the main reason for growing student loan balances, the report states that slow loan repayments have recently become the primary driver.

"Over the next few years, the combination of slow repayments and elevated, if no longer growing, levels of new borrowing will likely fuel further increases in outstanding debt," the authors of the report write.

In recent years, the number of students enrolled in higher education has declined and the cost of attending college has stabilized relative to people's incomes, Moody's analysts said. But borrowers have been slow to pay back their debt, meaning student loan balances will keep growing over the years.

Over the past decade, the aggregate annual net student loan repayment rate — meaning the amount of existing balances eliminated each year — has averaged about 3%, according to the Moody's report.