"... the Ainu territory, which comprised almost 95 per cent of the island's land area, was not part of Japan and its Ainu inhabitants were not Japanese. Japanese could make seasonal trading or fishing forays into the Ainu territory, but they could not settle there permanently. Ainu were similarly prohibited from travelling outside their own area ..." (Howell, 1994).
The Japanese 'invaded' the land of the Ainu in much the same way that Canada's aboriginal peoples were invaded by the Europeans. What's ironic is that the Japanese prohibited the Ainu from going wherever they want despite the fact that they had their land taken over that originally belong to the Ainu.