Is it possible to avoid using material under certain conditions?

Yes, it is possible. In Flutter, widgets like MaterialApp and others from the material package provide styles and behavior in the Material Design style, but if you want to create an app with a different style or a minimalist interface, you can use widgets from the cupertino package (for iOS-style) or simply basic Flutter widgets without Material. For example, if you don't need an AppBar or standard Material buttons, you can do without MaterialApp and use WidgetsApp or just Scaffold without Material.

Example without MaterialApp:

import 'package:flutter/widgets.dart';

void main() {
  runApp(Directionality(
    textDirection: TextDirection.ltr,
    child: Center(child: Text('Hello, Flutter!')),
  ));
}

Thus, using Material is not mandatory if the design and functionality do not require its components.