I think that you can, as long as the entire project as a whole, when distributed, is licensed by GPL, even if all of your own source files use MIT license. (Since it is compatible, it can be done.) If you later remove CGAL then you do not need GPL, I think, and can distribute the version without CGAL as MIT license. (However, I am not a copyright lawyer so I don't know, but I think this is correct.)