Having led for the majority of the second half the Ospreys appeared to have let the game slip after the English club earned a penalty try in the closing stages.
But Morgan Allen thundered over with less than a minute to spare to hand his side a dramatic victory in their opening LV=Cup clash.
Billy Burns fired the visitors into the lead but Morgan quickly hit back for the hosts.
Gloucester were beginning to exert their dominance at the scrum and they were rewarded with a second Burns penalty but the Ospreys again responded with a Morgan drop-goal.
The Cherry and Whites continued to put the Ospreys under pressure and the hosts paid the price for their indiscipline when James King was sent to the sin-bin for coming in at the side.
Gloucester instantly made their numerical advantage count as Thomas crashed over. They went to the corner after King was shown yellow and launched a rolling maul. Tommaso d’Apice broke free and slipped a superb offload to send Thomas over, with Burns converting.
But Morgan continued to chip away and he slotted another ugly drop-goal and a second penalty to reduce the deficit to a single point.
And the pint-sized fly-half put his side in front with the last kick of the half to bring the scores to 15-13 at the interval.
Morgan was off-target within minutes of the restart but the Ospreys were in control of the battle for territory and possession and he made amends on 56 minutes.
Sam Davies, the son of Gloucester director of rugby Nigel, dealt another blow to the Cherry & Whites to stretch the hosts’ lead to 21-13 and they appeared put the result beyond doubt when Tom Grabham slid over.
But Dan Murphy bulldozed over from close-range to give the visitors a glimmer of hope and they surged into the lead courtesy of referee David Rose, following a powerful rolling maul.
It appeared to be a knockout blow to the Ospreys but the Swansea-based outfit came storming back. After opting to go to the corner from a penalty Tom Habberfield put Gloucester on the back foot and after carrying on the move Allen was powered over by the Ospreys pack for the win.
