Is adding methods in a subclass a poor design?
No, it is not, but your claim is incorrect:
As to access m3 we have to launch B and this requires the use of the operator instance:
In your example, you are calling m3 directly after instantiation, so the if condition is always true:
A a = new B();
if(a instanceof B)
{
B b = (B) a; b.m3();
}
It can be easily rewritten as:
B b = new B();
b.m3();
A a = b;