Sir Tasker died in September after a short illness at the age of 88.
Representatives from the worlds of rugby, the law and the military will be at the service for Sir Tasker who was awarded the Victoria Cross in 1944 while serving with the Welch Regiment in Normandy.
Sir Tasker became the 46th President of the Welsh Rugby Union in 1993 and was a devoted supporter of Glamorgan Wanderers Rugby Club, who have postponed their Saturday fixture because of the memorial.
During his legal career as a barrister and then a judge he rose to become the Deputy Chief Justice.
Speakers at the memorial service include the WRU President, Dennis Gethin, Lord Justice Pill and Sir Ronald Waterhouse.
The service for invited guests takes place from 11.00am.
