I originally tried adding the GTM using Glue for Yoast SEO and it did track the pages. However, that resulted in 2 AMP validation errors:
The proper way of implementing the tag as of right now is via amp-pixel or amp-analytics. I used amp-analytics tag myself. First you would add this in the head:
<script async custom-element="amp-analytics"
src="https://cdn.ampproject.org/v0/amp-analytics-0.1.js"></script>
And then the tracking code goes in the body inside this tag using JSON:
<amp-analytics type="googleanalytics" id="analytics1"></amp-analytics>
Full details can be found here:
Adding Analytics to your AMP pages