Some banks ran out of cash. At others police were called in to manage queues of anxious customers hoping to change their savings for legal tender.
The surprise government move is aimed at tackling corruption and tax evasion.
But many low-income Indians, traders and ordinary savers who rely on the cash economy have been badly hit.
Banks were shut on Wednesday to allow them enough time to stock new notes following Tuesday night's announcement. There are also limits on cash withdrawals from ATMs.
The two notes accounted for about 85% of the cash in circulation.
The BBC's Geeta Pandey in Delhi says some banks extended working hours to deal with the rush on Thursday, and hired extra temporary staff.
Bank officials told the BBC that they had also brought in extra cash to deal with the situation - things had generally gone smoothly apart from the police having to deal with sporadic fights that broke out among customers.