Man United were largely poor against their Championship opponents and they were eventually knocked out on penalties after a goalless 120 minutes.
The defeat meant United have been knocked out by lower league opposition for the second consecutive year after MK Dons dumped them out of the competition last term.
Van Gaal revealed his disappointment with the result in his press conference on Friday.
“When you come to the match and see how we have played, you see how Boro played, also in a fantastic way. It is of course not so nice for us, but I think that the fans have enjoyed it,” van Gaal said.
“But we are out of the cup and you know that the next day is always like that. In the first round, we were knocked out by MK Dons, now we were nearly in the quarter-finals.
“But I have to rotate because we have so many games and the players cannot recover. All the managers agree about that because they are all doing it.
“When you see the history of Boro, they beat City 2-0 at City last season, but we were 0-0 and we could have won. But we lost and as a manager of Manchester United, you know you shall have a hard time.”
The result against Middlesbrough was desperately disappointing, doubly so because the Capital One Cup provided a good platform for the likes of Jesse Lingard and Andreas Pereira to impress on. Van Gaal will be keen for his side to bounce back against Crystal Palace this weekend.