Viewing a single comment thread. View all comments

NoKatanaMana t1_iynanv7 wrote

>We introduce a simple function torch.compile that wraps your model and returns a compiled model.

This will be interesting to try out and see how it develops.

110

gambs t1_iynencv wrote

I've been using JAX recently and the compiler has kicked my ass in so many ways, it is very hard to get used to and there are many things it just straight up prevents you from doing

Will be interesting to see if PyTorch can make a more enjoyable experience on this front

60

Desperate-Whereas50 t1_iynp7vk wrote

Imho you need functional programming or undefined behaviour (like in C/C++) to get high optimized code. Undefined behaviour is more pain than functional programming, so i doubt it.

Edit: And even C/C++ compilers like gcc have tags for pure functions to improve optimizations.

15

gambs t1_iyns2fy wrote

It’s not just functional programming, but for instance, you have to use jax.numpy instead of numpy when compiling, but also not every function from numpy is implemented in jax.numpy, and other issues like that

22

Desperate-Whereas50 t1_iynukpa wrote

I only have the informations of your link. So I dont know about the other issues you talk about.

But if you set for the functional paradigm it is obvious that you need some jax.numpy and that jax.numpy can not implement every numpy function. Numpy and some of its functions (like inplace updates) are inherent non functional. I cant imagine an other way to fix this.

6

kc3w t1_iyosur2 wrote

> Imho you need functional programming or undefined behaviour (like in C/C++) to get high optimized code.

That's not true, see rust.

7

Desperate-Whereas50 t1_iyow7w7 wrote

I am no rust expert therefore convince me that I am wrong, but that is only true if you dont use unsafe blocks. This would exclude using CUDA and as far as I know in some cases you need unsafe blocks to get C like performance.

But even if I am wrong and no undefined behaviour is needed. Even Rust has a pure function attribute to improve optimizations.

It just makes sense to use this improvements in libraries like pytorch/jax. Especially since mainly mathematical operations are performed that are pure functions anyway.

3

Craksy t1_iypi2h6 wrote

I'm no expert either, but you're right that using CUDA requires use of unsafe. I believe kernels are even written in C through macros.

However, using unsafe does not necessarily mean UB. You preferably want to avoid that regardless. And UB is not the only way a compiler can optimize. Unsafe code simply means that you are responsible for memory safety, not that it should be ignored.

I don't know, you're talking about UB as if it was a feature and not an unfortunate development of compilers over the years.

In fact, Rust made it very clear that if you rely on UB that's your pain. Don't come crying in a week when your shit does not compile anymore. No guarantees are made, and no extra consideration is made to maintain compatibility with programs that make up their own rules.

5

Desperate-Whereas50 t1_iyqiqqi wrote

>However, using unsafe does not necessarily mean UB. You preferably want to avoid that regardless.

>Unsafe code simply means that you are responsible for memory safety, not that it should be ignored.

Maybe I am wrong but I think you misunderstand UB. Of course you want to avoid UB and have memory safety in your code/executable because otherwise you can not argue about the program anymore. But you want UB (at least in C/C++ the language I work with) in your standard. UB is more like a promise of the programmer to not do specific things. The compiler assumes the code contains no UB and optimizies like that. See for example signed integer overflow. Because the compiler knows this is UB and the programmer promised to not allow it he can use better optimizations. Rust does not have this "feature" in safe blocks and produces less optimal code.

>And UB is not the only way a compiler can optimize.

I would not disagree about that. But if you want the last .x% of performance increase than you need it too. Especially if you want your language to work on different systems. Because even Hardware can have UB.

The only other option (as far as I know) you have to get some comperable (with UB assumption) performance is to rely on other assumptions like functions have no side effects etc.

>I don't know, you're talking about UB as if it was a feature and not an unfortunate development of compilers over the years.

As language specification it is like a feature. In the binary it is a bug. I have read enough discussions of UB in C++ threads to know that a lot of C++ developers dont see UB as unfortunate development of compilers.

>In fact, Rust made it very clear that if you rely on UB that's your pain.

By the way this is the sentence why I think you that you misunderstand UB. As mentioned: You should never rely on UB you promised the compiler to avoid it. And by avoiding it the compiler can work better.

1