Designing a proxy class in CPython for a class in uPython

[kc64]

I have a class in uPython that I want to interact with in CPython. self.obj is the belay device. Rather than issuing commands like self.obj('klass.attr') to get the value of the attr, I'd rather type klass.attr in CPython and have it proxy for me. Rather than self.obj('klass.__dict__') or self.obj('klass.__class__.__dict__'), I prefer to type something like klass.get() which makes the call for me to get all the attrs of the uPython object. On CPython, my __getattr__ method looks like this.

def __getattr__(self, name):
    if name in ['__belay__', '__isabstractmethod__']:
        raise AttributeError
    return self.obj(f'{self.name}.{name}')

This works. But when I try to implement a __setattr__ method, I get the typical recursive errors issues.

Has anyone tried something like this? What method(s) did you use to successfully proxy a class in C for a uPy class?

[BrianPugh]

As an experiment, I just wrote up a ProxyObject here that seems to work. Should I add this as part of the Belay library? It has some quirks, but it satisfies your use-case.

[kc64]

Thank you for such outstanding engagement and support. It is uncommon.

This is very interesting. However, I'm going to have to study how object.__setattr__ and object.__getattribute__ work. I don't yet understand how object. works in this case.

I do have a concern which might be unwarranted. At the CPython REPL:

>>> dev.klass.foo
1
>>> dev('klass.__dict__')
{}
>>> dev('klass.__class__.__dict__')
# ...throws an exception here but gives the result:
{'some_method': <function some_method at 0x2000a760>, 'foo': 1, '__module__': '__main__', 'bar': 2, '__qualname__': 'Klass'}
>>> dev.klass.foo = 10
>>> dev('klass.__dict__')
{'foo': 10}    # sets the attr on the instance
>>> dev("klass.__class__.__dict__.get('foo')")
1   # but there is still a foo on the class

[BrianPugh]

I don't yet understand how object. works in this case.

A relatively terse explanation:

1. Basically in python, foo.bar(baz) is syntatic sugar for bar(foo, baz).
2. Even though we overwrite our class's __getattribute__ method, we can still access the "vanilla" method via the fundamental building block object class. We can apply that vanilla method to our customized class.
3. The same applies for __setattr__, this way we can avoid infinite recursion as we get/set methods.
4. This still leaves a few quirks. For example, how does __call__ not trigger our __getattribute__? Well, some methods are special.

I do have a concern which might be unwarranted. At the CPython REPL:
...
# but there is still a foo on the class

This is a sharp corner of python involving class-vs-instance attributes, as well as mutable vs immutable objects. Here's a demo/explanation:

class Foo:
    int_a = 1
    mutable = []

    def __init__(self, int_b):
        self.int_b = int_b


foo = Foo(2)

# First, lets demo `int_a`; these all point to the exact same object.
print(foo.int_a)  # 1; `int_a` doesn't exist in the `foo` **instance**, so it looks it up in the `Foo` **class**
# The following 3 prints are different ways of saying the **exact same thing**
print(Foo.int_a)  # 1
print(foo.__class__.int_a)  # 1
print(type(foo).int_a)  # 1

print()

# Now, lets set the instance `foo`'s int_a
foo.int_a = 100
print(foo.int_a)  # 100
print(foo.__class__.int_a)  # 1

# When we set `foo.int_a`, it wrote the value *just for that instance*.
# Now when we do the `foo.int_a` lookup, it's looking at it's own `int_a`, **not the classes**

print()

# But now lets see what happens when we modify a mutable object, such as a list:

foo.mutable.append("I added this string to the instance!")
print(foo.mutable)  # ['I added this string to the instance!']
print(Foo.mutable)  # ['I added this string to the instance!']
# when we did get `foo.mutable`, that attribute doesn't exist in the `foo` instance,
# but it does exist in the `Foo` **class**, so we got the list from `Foo.mutable`.
# Then, we did the `append` operation, which adds an element to that list (it's mutable).
# This means that, unless directly overwritten at the instance-level, all objects of Foo
# will share the same `mutable` list.

[BrianPugh]

So actually, slight addendum:

type(obj) does NOT trigger __getattribute__, while obj.__class__ does trigger __getattribute__, so in this very-specific and quite-small-niche-of-python, they are not identical.

[kc64]

Many thanks! I have learned a lot from you.

The ProxyObject works great. I think it would make a nice addition to the Belay library.

[BrianPugh]

sounds good! I'll try and formalize it a bit more this weekend.

[kc64]

A couple observations from my testing:

1. Executing dir(proxy_object) throws an exception.
2. Adding methods to a subclass of ProxyObject does not work.

I'll keep testing.

[BrianPugh]

if you're subclassing the ProxyObject, you might probably just be better off subclassing device and making appropriate tasks and/or properties that call device. At some point there's overlap in functionality.