Plugin demonstrating how to inject GTM for the AMP plugin, in response to support forum topic: https://wordpress.org/support/topic/amp-with-google-tag-manager/

amp-google-tag-manager.php

<?php
/**
 * Plugin Name: AMP Google Tag Manager
 *
 * @package   AMP_Google_Tag_Manager
 * @author    Weston Ruter, Google
 * @license   GPL-2.0-or-later
 * @copyright 2019 Google Inc.
 *
 * @wordpress-plugin
 * Plugin Name: AMP Google Tag Manager
 * Description: Demonstration for how to add Google Tag Manager (GTM) to an AMP page in WordPress.
 * Plugin URI: https://gist.github.com/westonruter/2ea25735be279b88c6f0946629d0240c
 * Version: 0.1.0
 * Author: Weston Ruter, Google
 * Author URI: https://weston.ruter.net/
 * License: GNU General Public License v2 (or later)
 * License URI: http://www.gnu.org/licenses/gpl-2.0.html
 * Gist Plugin URI: https://gist.github.com/westonruter/2ea25735be279b88c6f0946629d0240c
 */

namespace AMP_Google_Tag_Manager;

const GTM_CONTAINER_ID = '______________________'; // 👈👈👈 This must be populated with your appropriate value.

/**
 * Print amp-analytics.
 */
function print_component() {
	printf(
		'<amp-analytics config="https://www.googletagmanager.com/amp.json?id=%s" data-credentials="include"></amp-analytics>',
		esc_attr( GTM_CONTAINER_ID )
	);
}

add_action(
	'wp_footer',
	function () {
		if ( function_exists( 'is_amp_endpoint' ) && is_amp_endpoint() ) {
			print_component();
		}
	}
);

// Classic mode.
add_filter(
	'amp_post_template_data',
	function( $data ) {
		$data['amp_component_scripts'] = array_merge(
			$data['amp_component_scripts'],
			array(
				'amp-analytics' => true,
			)
		);
		return $data;
	}
);
add_action( 'amp_post_template_footer', __NAMESPACE__ . '\print_component' );

That's great but where shall I put the file ?
I tried to put it in wp-content and there are no reference to www.googletagmanager.com appearing on my website ...

You put this file in wp-content/plugins and then activate the plugin in the WordPress admin. Make sure to update the GTM_CONTAINER_ID.

Thanks, when I tried, I saw the amp-analytics code in the header, and the GTM code in the footer.
However, since GTM was not firing and after investigating, there was a 403 error when loading AMP.js due to CORS error:

2www.googletagmanager.com/amp.json?id=GTM-MyContainerCodeHere&__amp_source_origin=https%3A%2F%2Fexample.com:1 Failed to load resource: the server responded with a status of 403 ()
example.com/:1 Access to fetch at 'https://www.googletagmanager.com/amp.json?id=GTM-MyContainerCodeHere&__amp_source_origin=https%3A%2F%example.com' from origin 'https://example.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
log.js:244 [AmpAnalytics ] Error loading remote config: https://www.googletagmanager.com/amp.json?id=GTM-MyContainerCodeHere Error: XHR Failed fetching (https://www.googletagmanager.com/...): Failed to fetch​​​
at $a (log.js:707)
at Wa.f.createExpectedError (log.js:356)
at xhr-impl.js:109
at async Promise.all (/fr/avantages-amp/index 0)
Ya @ log.js:244
log.js:244 [AmpAnalytics ] No request strings defined. Analytics data will not be sent from this page.
Ya @ log.js:244
log.js:244 [AmpAnalytics ] No triggers were found in the config. No analytics data will be sent.

I also checked that it works with AMP disabled and the "normal" GTM setup.
I also tried to use wp_body_open instead of wp_footer and it did not work either.

I'm not an expert on GTM. I only made this plugin to output the required AMP markup. For help with why GTM may not be working properly, you'll need to reach our to GTM support.

Installation instructions: https://gist.github.com/westonruter/6110fbc4bef0c4b8c021a112012f7e9c