Virtual environments doesn't work for the interpreter itself. Not only that some of the packages will need to build c extensions and they use different versions of the same library which might break.