Erik Schnetter: Okay, good morning I want to briefly introduce spec the package manager I have prepared some slides actually from my previous presentation that are just going to warm up again, we can go through this quickly.
Erik Schnetter: And then I will going to give an interactive presentation and just to introduce what spec is about it's about a package manager and anytime you install software it's.
Erik Schnetter: really difficult, of course, you all know that, and in particular on htc system and you install the Einstein a tool kit you have to install all the dependencies the hdfs and mpi and fw and dependencies.
Erik Schnetter: And then it's to all work with the same API version, and the same compiler and be consistent and that spot spec is helping about.
Erik Schnetter: Now I don't know about your background is and how experienced, you are and where the emphasis here should be on how much I should philosophize about why it's back, this is a good idea.
Erik Schnetter: or just show you interact with the how to actually do it so i'll just start with the slides here and yeah.
Erik Schnetter: don't be intimidated by the slides please ask the questions I guess you all know that building from source is difficult and particular everything needs to be consistent.
Erik Schnetter: If you have used a particular version of mpi and then use html5 it needs to be the same version of mpi all the software needs to be built with the same compiler or almost the same compiler to remain consistent and so on.
Erik Schnetter: and various things that make it difficult, and if you know Python and Python it's much simpler you do pip install or condo install Conrad.
Erik Schnetter: And then it's all there and in C c++ fortran these old languages it's much more difficult.
Erik Schnetter: And that's what spec is here to actually help and Steve and I were using spec to install software on htc systems, it makes it much easier to get started with a new htc system.
Erik Schnetter: And I recently showed another student how to get started using spec and it was quite easy for them as well, so I thought i'd to spread the knowledge.
Erik Schnetter: And, in the end, maybe in the future, we can begin to use back in general.
Erik Schnetter: There are many hbc centers that have started to use back to install the software that has if they install html5 they wouldn't build it manually they just do the equivalent of spark installed hdfs with a particular mpi version so it's really a professional tool.
Erik Schnetter: and
Erik Schnetter: A spec is also part of the do E.
Erik Schnetter: X or scale computing project, so they have ci infrastructure, where they're built hundreds of packages every day with various compilers and various systems and so on.
Erik Schnetter: Just to make sure that everything works, but this project as at the moment, really, a lot of momentum and a lot of future.
Erik Schnetter: And it's quite flexible, a lot of features and I can only scratch the surface today and I mainly want to show you how I use it and then, when you've seen it maybe to get interested in as well.
Erik Schnetter: yeah.
Erik Schnetter: So you're the obligatory pointer to the documentation spec read the docs I O, and we felt the spec developers and I happen to be one of them I.
Erik Schnetter: want to price for that as an innovative software, because the.
Erik Schnetter: simplicity of it really makes a lot of different.
Erik Schnetter: And the principal installing software is as easy as typing spec style mx and then you wait a few minutes on a slow hbc system, you might have an hour.
Erik Schnetter: And then you have the current version of emmerich's available and there's really much simpler than figuring out how to download it and build and install and so on.
Erik Schnetter: Now, because I assume, most of you know how to install software and skipping most of the background Weiss back is a good idea and i'm going to the meat of things.
Erik Schnetter: How I use spec for the Einstein tool kit now what often happens to me is that I install some software for the Einstein toolkits on a particular system on in the system.
Erik Schnetter: And then other people come along collaborators students call supervised, and they want to use the same software, so they point to my directories, but I installed software.
Erik Schnetter: And that is really inconvenient, because then when i'd want to change something install a new version at breaking their setup and it's really inconvenient, so I really think everybody should install the software that's really the way to go.
Erik Schnetter: So the way I do it, I do spec not to install the Einstein took it rather than to principal and principal as well, but just all the dependencies.
Erik Schnetter: And spec installed it's packaged into its own directory, and that makes it really easy to replace the package within your version, without having to re install everything.
Erik Schnetter: And it's not one directory for html5 but it's a different directory for the package diversion a set of options and so on, as part of the package so you can have five different versions of html5 installed at the same time and there's no.
Erik Schnetter: No way they confuse each other and, later on, I create a view and, if you are essentially a set of symbolic links.
Erik Schnetter: Where there is one directory the view directory that contains these links to all the other installed packages.
Erik Schnetter: And spec make sure it's a self consistent set of packages, so if some packages define depend on mpi is the same API version, they all use the same compiler and so on and so forth, and then in the cactus option this it just point to the view directory and that's approximately it.
Erik Schnetter: And unless their questions, I want to skip the clinic clinic the preliminaries and then actually give a demonstration how I use the spec and practice.
Erik Schnetter: Okay, very good let's get started i'll stop the screen share and i'll open a new directory sorry and open to him.
Erik Schnetter: That i'm going to share i'm going to log in to symmetry the local hbc system.
Erik Schnetter: I assume you can see the screen is the font size right yes.
Beyhan: it's perfectly.
Erik Schnetter: No one complains i'm going to log in here and.
Erik Schnetter: It takes too long, creating a new directory, and in this directory i'm going to install things and i'm going to start from scratch here because I have notes action sheet.
Erik Schnetter: So the general idea is i'm going to download and it starts back here, using spec i'm going to download and install your packages and from then we can build the Einstein pocket so.
Erik Schnetter: let's download something and I have to teach it here i'm willing to share that later to downloads back you can go to the spec web page, of course, that is, that URL here.
Erik Schnetter: And going to download it the command that is recommended at this one it's just the URL yeah.
Erik Schnetter: that's apparent in you get to feature many files equals true for repositories that contain many files, I have never seen it before but spec recommended i'm going to use it and i'm going to download version zero 17 to have spec.
Erik Schnetter: know what is from now great.
Erik Schnetter: sorry about that.
Erik Schnetter: yeah let's see how it goes over there i'm going to use a different system desktop better than you actually get a live DEMO that will be much more interesting.
Erik Schnetter: Okay, so this is where I installed spec yesterday and I.
Erik Schnetter: Okay Well, this is not going to work.
Erik Schnetter: let's use another system and let's use debug you why not right.
Steven R Brandt: Right now.
Erik Schnetter: I have a shortcut that makes it easier to look into other systems so here, are you let's hope I have enough quarter here.
Steven R Brandt: You know if you don't you can always install a slash type system.
Erik Schnetter: yeah now i'm going to going to do it here on people yeah that's more honest anyway because anything that goes wrong is actually real anywhere i'm going to install version zero 17 to have spec.
Erik Schnetter: Okay i'm going to clone that thing and then check out the tech maybe that works okay usually i'm using the development version of spec but they're having a big internal be arranging where that hashes they're using.
Erik Schnetter: More tight so.
Steven R Brandt: I think.
Erik Schnetter: I think.
Steven R Brandt: i'm sorry I think if you'd left off the tags slash it would have worked.
Erik Schnetter: Oh okay that's possible.
Erik Schnetter: Oh attack is probably just V zero 72 tax is not actually part of the tech I got confused.
Thank you Steve.
Erik Schnetter: yeah and The hope is that I have no idea of like downloading spec is so slow, I mean they have.
Erik Schnetter: Thousands of package description package recipes but each package recipe it's just a small Python script a few kilobytes and that's the thing itself, which is not very large as the Python code, so I don't know why spec is so large I don't know what is in there, he presents of test cases.
Steven R Brandt: could be the history.
Steven R Brandt: Maybe if you use def one.
Erik Schnetter: Maybe.
Erik Schnetter: And i'm checking out to this particular version.
Erik Schnetter: Very good, and now we need to actually get this back executable, you said that you environment bibles and that's a script in the specter to you that does that i'm going to run this.
Erik Schnetter: And now I have spec available I have spec help.
Erik Schnetter: And you see all the commands that you can hear not not too much so let's do the first steps, just to see what's happening here so expect find lists all the install packages.
Erik Schnetter: Zero installed packages that is properly correct now i'm going to install and deadlift just as a warm up because that's an easy quick packages easy to install so the spec install is that.
Erik Schnetter: And that should not take a few minutes because spec needs a few other things external packages and it's now supposed to download these as well.
Erik Schnetter: For the first time you're on spec it probably takes minutes you can run spec bootstrap or something manually if you want you to do that and, obviously, that it's slower that happens, just the first time, so don't worry about it, if you're using it every time.
Erik Schnetter: And particular as one thing when you install packages, they have dependencies and there are certain constraints that one version of one package doesn't work with another version and so on.
Erik Schnetter: And to handle that correctly spec as a constraint solver built in and uses klingler there is a professional constraints over and that's one of the things it's downloading and installing now behind to back yourself you're raising your hand.
Yosef E Zlochower: Yes, i'm just i'm thinking when I when you build the toolkit often you're using something not the not the system compiler so.
Yosef E Zlochower: How do we specify that let's say to build everything with say I don't know your favorite version of the Intel compiler yes.
Erik Schnetter: I get to that later spec is a special provisions for that and then that's a big part of what i'm going to describe later just doing the warm up i'm going to use the system compiler here.
Erik Schnetter: Anyway, so installed it, so I mean the waiting was just the bootstrapping and then, when the message installing appeared that it's actually started to install something so it didn't find a binary package you downloaded it.
Erik Schnetter: knows where to get it doesn't need any purchase and install it and it installed it into this directory, you know how the directory name is pretty long has a.
Erik Schnetter: package name the version number and the long hash so in the install 10 different versions of that little bit different compilers the different versions of different the FLEX that all different hashes.
Erik Schnetter: And photogs back find again to see there's one package installed, and we know that system is a Linux system uses the right tech seven as a cascade lake cpu that's a good one, more than one and it uses GCC 930 to install it.
Erik Schnetter: Now.
Erik Schnetter: One problem is when you're installing software is you can bring everything yourself, except for mpi.
Erik Schnetter: So you can install your own compiler if you want to you can install your own html5 she could install your encoder and that all works fine.
Erik Schnetter: But it doesn't work with mpi because mpi needs access to the fast interconnect.
Erik Schnetter: And it has hardware dependencies mpi libraries, need to be installed by system administrators, they need to know about kernel modules for drivers specific other libraries are fair for shared memory.
Erik Schnetter: In your band So if you installed your own mpi as basically impossible to use the infinite band, you probably forced to use tcp so.
Erik Schnetter: One thing we are going to have to do is tell spec what system mpi they're using and half point respect to that one and.
Erik Schnetter: I, and this tutorial i'm also going to point it to one of the compilers that exists on the system.
Erik Schnetter: Not necessarily the default compiler principle, you could install the on compiler media into a compiler spell if you have a license.
Erik Schnetter: But that is currently a bit complicated so before we begin to install the software you're going to have to do steps.
Erik Schnetter: you're going to have to choose what compiler and what system mpi be using and choice of system mpi probably also determines the choice of compiler so let's use let's choose first but mpi you're going to use yeah i'm just going to type module avail to see what mpi versions there.
and
Erik Schnetter: Steve you know, the system is there a particular version, you want to suggest that would be a good one.
Steven R Brandt: um i've i've used em pitch Intel before I don't know that i've done it specifically with spec like this, but hey what the heck but dangerously.
Erik Schnetter: Okay i've heard rumors that stock mph might not be efficient, with any band that advantage open mpi is more efficient, but OK let's use that one here so First, we need to.
Erik Schnetter: So one of the things we need to do is find out where this and purchase installed, so we can point specter and we're also going to have to choose the compiler.
Erik Schnetter: And I think i'm just going to use this compiler htc 9.3 because that's convenient one so to find out the compiler is a command spec compiler list that I can run.
Erik Schnetter: This looks at the system compilers and it found these compilers there might be more compilers in the system, and then you can point back to it, but this otter search is good enough so i'm going to use this one in a very happy that's back on it, I didn't know that ahead of time because.
Erik Schnetter: I didn't do the repair this for the tutorial yeah.
Erik Schnetter: OK now i'm going to create a certain yama file and this file tell spec what packages, I want to install and but compiler supposed to use and about this external package so.
Erik Schnetter: and, later on, I can send around a template for that.
Erik Schnetter: Actually, I prefer to template here so just going to copy and paste that are going to have to modify it.
Erik Schnetter: significantly.
Erik Schnetter: So their head park here is back event, I can tell us back about pre existing packages on the system, and this is not for this system right it's for a different system so ignore all the details.
Erik Schnetter: And then, when back install something installs so called specs and respect as a package does all the options that you need for the package.
Erik Schnetter: So here are the specs i'm going to install that i'm going to install just a few packages because that's kind of the bare minimum, you need for cactus i'm going to install.
Erik Schnetter: it's going to commented out and NPI opened last also commented out in that you can easily install other things fft w X on.
Erik Schnetter: conquer translation means and that prevents back determines all the dependencies that these packages here require.
Erik Schnetter: it's going to make sure it's a self consistent set of packages I don't want six independent packages so, for example, then html5 requires an mpi spec is not free to choose which one.
Erik Schnetter: It needs to be consistent with this one, and so on, so that later on, I can use all these packages together.
Erik Schnetter: and few that tells spec that later on, it should create this view of these packages and all its dependencies where everything is installed siblings to the actual package cash okay.
Erik Schnetter: Now this is the wrong directory i'm going to have to find out what the actual directory is very it is i'm just going to copy this one here.
Erik Schnetter: So in this directory, yes, they are going to have that.
Erik Schnetter: No.
Erik Schnetter: Of course I didn't I don't remember OK OK.
Erik Schnetter: let's type module load and then this package here.
Erik Schnetter: And then rich mpi run.
Erik Schnetter: Just find the rich and the ICC.
Erik Schnetter: There okay.
Erik Schnetter: And then module unload and pitch to get rid of that again, and this is the past two which i'm going to point spec.
Erik Schnetter: So this is the prefix going to put this one in here just the prefix you this is directory where the average is installed that I want to use, and this is unhampered and it is version three dot three dot two.
Erik Schnetter: And later on.
Erik Schnetter: When I installed it I say I want and pitch version 3.3 Point two.
Steven R Brandt: And you miss miss miss one thing they ever sold an open mpi.
Erik Schnetter: Thank you.
Erik Schnetter: And so, this means I tell spec about some special thing about amperage and there's an external version of MP3.
Erik Schnetter: So if I installed and pitched it to spike is going to change this one it's not going to build anything it's just going to reuse this one.
Erik Schnetter: If I installed and pitched 333 down here and spec would build it from scratch and presumably that version would not know anything about infinity abandoned would be much lower.
Erik Schnetter: So for all packages we don't really cover tuition just for mpi you need to point to a system mpi Oh, and you have is say.
Erik Schnetter: That just for fun, the last version 9.3 yeah so just for fun I tilts back, I want to install GS a library with GCC 930.
Erik Schnetter: And then, all the other specs to get infected perfect and it's probably choosing GCC 930 as well.
Erik Schnetter: If I left that out spec would be free to choose any compiler and by default, it would probably use she sees 3930 smell but let's keep that in here.
Erik Schnetter: And of course you can add this tag GCC 9302 all the other packages and they're convenient ways to make it automatically apply all the packages and so on i'm going to skip all that because that's too complicated, so this.
Erik Schnetter: is something that you need to have need to describe in the beginning, probably for every system that's called an environment definition and now let's go on so first.
Erik Schnetter: i'm going to ask spec to find out about all the dependencies that it needs to install as well.
Erik Schnetter: So, because we defined an environment, I need to spec aware tell spec where that environment this in here in this directory.
Erik Schnetter: And then i'm going to use the command concrete concrete is means that spec the look for the flesh out all the spec can build tell me gory detail for all the packages installs.
Erik Schnetter: That is this one here so GS la it's easy it's going to install DSL version 2.0 w lock has a few dependencies it's going to install these as well the PC I access that request lip tool that class, and for that requires and so on and so forth.
Erik Schnetter: And curses oh right which w has some interactive thing and pitch it's not dependencies because it's just using the system and pitch I hope.
Erik Schnetter: And that clip is also that one now i'm going to run spec install.
Erik Schnetter: And that should install these packages remember in the environment file and commented out hdfs and.
Erik Schnetter: Open blast because these take a long time to compile but this one here should just take five minutes to compile.
Erik Schnetter: And now psl is being installed doesn't require any purchase apparently when you install G is l it's not just configure make make install the need to talk to a recon spec know that now it's compiling it.
Erik Schnetter: And it should be temporary, some.
Erik Schnetter: Okay we're not going to watch all of that, I want to show you a few other things, but maybe let's just watch one or two packages.
Erik Schnetter: Okay yeah Steve if you're interested Maybe you can wind up in the background to see how busy the head note, yes.
Erik Schnetter: So actually I can do that.
Erik Schnetter: Very quickly, I can just run up and oh yeah look it's compiling many things at the same time that started in parallel very nice.
But still busy.
Peter Diener: So does it ever happened that something goes wrong during.
Erik Schnetter: Oh yes.
Erik Schnetter: compilation and add.
Peter Diener: them yeah.
Erik Schnetter: What do you do, then so.
Erik Schnetter: I mentioned earlier that spec recipes know what is compatible with what and sometimes it's not really clear, for example, i'm using ad has to and that doesn't work within your version of html5.
Erik Schnetter: And what I usually do at just add in version numbers and say, well, using earlier version of ideas to visit different version of html5 get around that and so on, and that works, most of the case, most of the time.
Erik Schnetter: And sometimes I say well on that system, I really don't care, I tried to use system ambridge I can do system open mpi and maybe that works.
Erik Schnetter: And when it's hot curve and I open a bug report and maybe one of the packages needs a patch and you can tell spec to apply a patch to a package with a building it, but of course that I mean.
Erik Schnetter: If the download the package find out what patch and create an action, so this is really tedious I mean once you add that level it.
Erik Schnetter: It becomes really tedious I hope you don't have to know how see how quickly all these things and stuff isn't it nice actually it's fast, because people use a system called suspect okay.
Erik Schnetter: same time, let me show you.
Erik Schnetter: So your typical package recipe just for fun.
Erik Schnetter: Oh, oh my before that.
Erik Schnetter: Let me show you this is essentially the spectral yama file that I use.
Erik Schnetter: On symmetry when I build a cactus so it's a bit more complicated than a short one year.
Erik Schnetter: I see what compilers I want here is what packages, I want, and you see half annotations with version numbers and so on and so forth, and then a usability matrix that I don't have to put this compiler annotation everywhere, but apart from that it's the same.
Erik Schnetter: and
Erik Schnetter: If you want to have it has a new package that you want to.
Erik Schnetter: install you can also create new packages.
Erik Schnetter: Quite a strong.
Erik Schnetter: Man my laptop yesterday, and I think I lost a few files.
Erik Schnetter: Maybe they just moved into dropbox.
Sorry.
Erik Schnetter: Anyway, this is a typical package recipe.
Erik Schnetter: or spec, and this is for the reprimand package that you might know it's not it's not very long currently you're seeing.
Erik Schnetter: yeah depends on you see probably 80% of it here it's a Python script it's for the reprimand packages uses mason as a build system.
Erik Schnetter: And Dave mason is not installed on the system of course back, but now they built it that has said, beginning.
Erik Schnetter: description on the to the homepage they could download things and maintain that means I maintain this recipe, not the package some versions developed version 1.3 1.4 has also been released there's a patch that you need to apply only for version 1.3 you can do that.
Erik Schnetter: That two variants you can install reprimanded different ways, you can enable Python or disable shared libraries, it has a few dependencies that need to be installed.
Erik Schnetter: And you need to set environment variables before you build into past certain options to miss on I don't want documentation and build Python API depends on whether you chose to Python very and not.
Erik Schnetter: The actual building uses mason default you don't have to describe any there.
Erik Schnetter: And the result, the libraries, that this creates is this lip reprimand something that so don't delay or something depending on the system if you have shared libraries and this finds that.
Erik Schnetter: So, creating a package recipes quite straightforward, if you have a modern package, of course, if you look at the package recipe for Lorraine, or something that is an unholy mess.
Erik Schnetter: Because lauren is a really old package, and you have to edit make files and so on, and doing that in a Python script the factual expressions and sounds a bit more complicated, but it makes much Lorraine much easier to install.
Erik Schnetter: Okay, we should be almost dampening and curses took a bit of time to build.
Erik Schnetter: yeah took almost two minutes to build it.
Erik Schnetter: Okay, I ran out of things, I wanted to show so right now we can just wait for two minutes after this done or you can ask questions.
Erik Schnetter: complaining about this and features.
Erik Schnetter: Okay, no, there was one more thing I wanted to show.
Erik Schnetter: When I ran computerise land spec created from that spectra gamble file aspect or lock file look at the difference in file sizes, is this 100 times longer.
Erik Schnetter: And they're not file contains all the details about all the dependencies that spec is going to install it Jason oyama file or something that contains all the details about all the packages and then you went to interrupt this back install here.
Erik Schnetter: And then we run it and it's just going to continue because it's going to build the same set of packages again Federico you have a question or comment.
Federico Gaston Lopez Armengol: Yes, higher So if I have many applications in the hbc system that you recommend to use to build a spark directory for each of them, or do you suggest to let's say they use different versions of.
Federico Gaston Lopez Armengol: packages.
Erik Schnetter: I recommend you have one spec install.
Erik Schnetter: And this back install has done a single build cash So if you happen to use the same package versions for different applications that can be the same, and then you have different you directories and if you directory points to the subset of the packages.
Erik Schnetter: Let me let's just go to the spec directory.
Erik Schnetter: where things are installs and up to spec it has here the subdirectory all the packages built with that compiler, and these are all the packages that currently built.
Erik Schnetter: And you can I mean if you install a different version of DSL or even the same version but different options it's going to be different hash so there's no clash, so you.
Erik Schnetter: have only one version of spec you install everything in here and then give you the point to a subset of these packages and actually.
Federico Gaston Lopez Armengol: All right.
Erik Schnetter: Since we are now done with a built let's look at the view directory save you been and then you see Todd is not God loves us a lesson that it's easier to see.
Erik Schnetter: These are all add a color scheme is not good, these are all the binaries from the things that are installed you see the DSL and.
Erik Schnetter: Double you know that we have, but also some binders from execs apparently some package at some compression library that can be binaries and that also install and if we delete the view, then of course the installed packages don't go away.
Erik Schnetter: So I don't know if it was that clear with the package cash spec installs everything and the view that you create.
yeah thanks.
Erik Schnetter: So let's change this back, but yama what else should we do.
Erik Schnetter: say the different version of setup.
Beyhan: Actually, Miguel has a question.
here.
Erik Schnetter: We go this.
Miguel Gracia Linares: Is Eric so, if I understand correctly, this also can be used in many like workstation right yes.
Erik Schnetter: It works everywhere, I use it on the MAC book as well.
Erik Schnetter: and
not sure, but window.
Miguel Gracia Linares: Okay, and everything here is installing work directory they.
Miguel Gracia Linares: So how about if you want to install in a different directory we just pointed out, to his pad or.
Erik Schnetter: Okay, we spec itself has its own package cash and it doesn't matter whether package cash lives, it should just be some valuable as a lot of disk space.
Erik Schnetter: And the view directory is that is the one that you would actually use so later on, you look at the view directory, and then you see a consistent set of packages.
Erik Schnetter: And appointed actors, only to the new directory and it doesn't matter if it's backwards, so in benign starts back I created a directory so spec and all the packages live there and then the view directory appointed in the value stack directories that.
Miguel Gracia Linares: Okay, thank you.
Erik Schnetter: yeah let's install the different DSL version, just to see what happens 2.6 see whether that works.
Erik Schnetter: And then i'm conquer tries again so that it would actually etc.
Erik Schnetter: So.
Erik Schnetter: Now it's going to be compiled version because it's not going to be compile anything else, and if you look at the package cash.
Erik Schnetter: When you see that they're going to be two different versions of esl and there's no class that's the.
Erik Schnetter: Old one and version 2.6 is the one that's currently being done, probably still yeah it's nothing in there still empty and if I want to switch back to 2.7 I can do that, and that would not require recombining anything, because all the packages installed here.
Erik Schnetter: Okay yeah now i'm not actually interested in that i'm just going to stop here, and if you type spec Finder Can you see all the packages.
Erik Schnetter: And you see, we install just five packages on all the other packages icon pci access that that just installed as dependencies, of the other packages and you might or might not care about them.
Peter Diener: Now for the packages you don't specify that gets installed, by default, others, those just the latest version or anything that is compatible with what you require.
Erik Schnetter: I think it's the latest version that is compatible.
Erik Schnetter: And as the default set of options and choosing these packages that's just the art that lies and the conquer transfer and then that constraints over.
Erik Schnetter: Very well, spec uses clinical because it can be very difficult to do that, and sometimes there are conflicts, for example, when one package requires html5 with mpi another class html5 without mpi and, of course, you get an error to conquer types of fields, because you can't build this configuration.
Erik Schnetter: And the spec recipes know about a certain set of versions for the packages.
Erik Schnetter: So you can of course edit the recipes yourself if you want, but if you upload a new version of spec it would know about a new set of packages.
Erik Schnetter: And then, when you run the contract is again, it could choose a different set of packages like maybe a more modern version of esl I think this aversion to 71 hour or whatever happens.
Peter Diener: I also have another question sorry, there are there any cases where you need final control over compile options for the various packages.
Erik Schnetter: Yes, out, for example on MAC the system compilers claim but cleanse no good because it has no fortran so I want to both more things with GCC.
Erik Schnetter: they're open MP things incompatible and so on, so for many packages, I need to manually say which to build the GCC because I want to use them in the accident took it.
Erik Schnetter: And, which to us with clang because they can't be built with GCC on MAC os.
Erik Schnetter: They have certain language extensions in the system header files and GCC doesn't handle them properly for like a see make I need to build with clank.
Erik Schnetter: I don't really care about how see makers build and then I need to have lots of individual tax what to build this email and started playing and what GCC.
Erik Schnetter: And this is similar problem with us employer compiler because it has just incompatibilities.
Erik Schnetter: So if you want to build something with the Inter compiler that probably other packages that you don't really care about probably see maker, so that still need to be appeared as GCC.
Erik Schnetter: And, in many cases spec this is clever enough to figure it out by itself, and then you have three or four different additional annotations okay actually I might have an example, if you give me a minute i'm going to look for some.
Erik Schnetter: Okay yeah he is a file.
Erik Schnetter: This one here there's a new compiler out there and vigor and the htc.
Erik Schnetter: is more comfortable for cuda and this nba hbc compiler that are also installed few years back.
Erik Schnetter: And it turns out that it can't build some packages, so I need to build them with GCC so have some extra packages here.
Erik Schnetter: And we see make a pie wheel and as that I don't care about these they're being pulled in this dependencies and explicitly specify that's back should build these with that particular GCC version.
Erik Schnetter: And it doesn't really matter what compels us to build see make because it doesn't end up with the executable and so on, and of course finding that is trial and error see MAC has a built our up and then I say about I sit back and then it works.
Erik Schnetter: Oh yeah and he had to choose a particular version of silo because the newer version of siloed don't have to do something with hdfs and so on.
Okay.
Erik Schnetter: Then i'm going to send around the files that i've produced here at just attach them to that.
Erik Schnetter: Announcement for my presentation, but of course there are a lot of tutorials and as a quick documentation for spec and you should definitely look at them.
Erik Schnetter: The main tricks that I used and that's not clear in the beginning, as that first doing stole all the dependencies vs back.
Erik Schnetter: But not the Einstein to get itself you build that the usual way with an option this option is just points to the new directory.
Erik Schnetter: Then you install all these things we are an environment it's not clear that that would be a good way, but that is you don't install the packages independently.
Erik Schnetter: Because we've installed them independently you wouldn't make sure that they work with each other, so you need to have an environment and you need to have this computation session together in the environment.
Erik Schnetter: And then later, you use a view because that's just an easy way to get a nice view of packages you don't want to point to a different directory for each of the packages that you installed yes.
Beyhan: very good question right.
Federico Gaston Lopez Armengol: So again, to have a sense of the of the of the workflow I see you have command line interface for spec and also, you have the young.
Federico Gaston Lopez Armengol: files that you showed So these are two ways of.
Federico Gaston Lopez Armengol: accessing the same package list of packages right which because when you when you start mentioning about the the file I start getting a sense like like a foot container.
Federico Gaston Lopez Armengol: container ization.
Federico Gaston Lopez Armengol: But.
Erik Schnetter: it's a bit like that.
Erik Schnetter: Because it an environment that we are defining here.
Erik Schnetter: And this environment had a set of packages that work together, and you can set if you buy the environment should be installed and set a few other things like compilers and so on and so forth.
Erik Schnetter: And it goes in the direction of docker file, but the docker file is a script that runs top to bottom, where this.
Erik Schnetter: Is an interactive interface so from that point of view it's better.
Federico Gaston Lopez Armengol: To be and they come in line to face.
Federico Gaston Lopez Armengol: Like.
Erik Schnetter: need that.
Erik Schnetter: I edited specter gamble file and then I call stack installs by concrete eyes and so on and so forth manually.
Federico Gaston Lopez Armengol: All right.
Erik Schnetter: You can also create environments, with a command line interface, you can create environment and at packages and remove packages and adoptions, I find that really tedious.
Erik Schnetter: Because it's much easier to edit that file and you see at once, but it seeing what you have to do.
Erik Schnetter: If you create an environment and add and remove it's, a thing that looks somewhere and you don't see where it lives on an htc system, whereas if you have a spectrum gamma Phi you put it in gifted version control and then you don't lose it.
Federico Gaston Lopez Armengol: So maybe a specific case, so if I add a Mr X, to the CMO file and on the other hand, if I write spaghetti style a rx that the effect will be the same right.
Erik Schnetter: Okay, if you added to this yama file nothing happens, but if it.
Erik Schnetter: Right to the file.
Erik Schnetter: And later one install and saying so that and read them and then it would install that version now if you just type spec install emirates then spec Willie installed emmerich's as a standalone package.
Erik Schnetter: it's not necessarily consistent with any other packages, so it might use an mpi version that's different than the mpi version use.
Federico Gaston Lopez Armengol: Oh, I see I see.
Erik Schnetter: And then you couldn't use them together, this is the main advantage of the environment.
Erik Schnetter: And the other thing is, of course, if I use this on one computer and you want to create environment, on the other, computer I just copy the file over and tweak it and then it's very easy to get started, mostly you have to edit the name of the view here.
Federico Gaston Lopez Armengol: Right now that's good thanks.
Peter Diener: So if you install several different versions of many different libraries and then, when you want to compile cactus, how do you tell it which ones to actually use.
Erik Schnetter: I create the view and the view here points to that version that was chosen for this environment, the viewer doesn't point to all the install packages.
Erik Schnetter: Points just to whatever the result of that quantization rest as the constraints over that uses something consistent here.
Erik Schnetter: All these packages descriptions and diversions that are listed here constraints and then it chooses preferences for all the dependencies in the other version numbers and it's one concrete set of packages and that's going to be in the view.
Erik Schnetter: And everything else, the other versions that just your maintenance back and it's just like a cash in spec actually are, if a types back find here.
Erik Schnetter: I kind of course also add install something can actually click on install sure I can just uninstall.
Erik Schnetter: Not a sensible thing to do, I can just do that, and then it had to put a stop this version, this is the hash it's going to and install it is and it's all ngs l that was built with that compiler and so on.
Erik Schnetter: And I can just remove it, and now of course it's not in the package cash anymore.
Erik Schnetter: Because it's bad because the view, and now it doesn't work anymore, well, it bro earlier because I said what did you install DSL to protect and then I bought it the compile for that, but that's a different story.
Erik Schnetter: And yeah I mentioned installing mpi spec is not a good idea, because mpi has dependencies and installing a compiler via spec is a bit more complicated that's probably going to be in the next version after major release of spec they're working working on that and if I.
Steven R Brandt: install compilers and spec all the time.
Erik Schnetter: Oh yes, I do that as well, but installing them automatically currently if you want to install the compiler you can do that, but if you want to use it.
Erik Schnetter: You need to have a second environment it doesn't work in.
Steven R Brandt: Korea okay.
Erik Schnetter: So I have one spectral yamagata installed the compiler in a second spent the Delaware use the compiler and other packages, which is why I didn't show it here and that's going to be easier, but with kuda, for example, although that looks like a hardware dependent package that is just something.
Erik Schnetter: That where is it, I can put here I just say, I want to be stole that version of code and the zip and spec downloads it and it works fine.
Erik Schnetter: Most of the time.
Erik Schnetter: So that means you have very few dependencies on the system.
Erik Schnetter: mpi epic and things like literacy and so on.
Erik Schnetter: Okay, other questions.
Peter Diener: So, you said you you, you are a spec developers, or what have you been doing for spec.
Erik Schnetter: Mostly.
Erik Schnetter: I create a package recipes the packages that are useful for the Einstein toolkit.
Erik Schnetter: I see and currently someone posted on Twitter a list of the top 20 spec developers and I was really surprised that my name was among that.
Erik Schnetter: I didn't think that I mean I also did some contributions to remain spec I think yeah Lorraine devastated, I wanted to install the room yes back and expected support CDs because really who use a CVs these days, I mean just update to a better system but there you go.
Erik Schnetter: And we really liked rain for neutron stars and relativity so I copied the code that supports SP en and added a piece of stuff and so and.
Erik Schnetter: i'm never going to get that time back I should send an email to the Lorraine developers, such as created as the version clone the repository that would have been much better.
Erik Schnetter: Better use of my time.
Erik Schnetter: Okay i'm just reading installing DSL 2.7.
Erik Schnetter: That he did earlier just for completeness just something called spike probably talk.
Erik Schnetter: They have other questions.
Erik Schnetter: Okay, very good, then, thank you for the interest and a half great hope that spec or other tools like that make it easier to install software oh I forgot something.
Erik Schnetter: there's another tool very similar to spec called easy build and I think it's about equivalent to spec I happen to run in to one of the spec developers early and this is why I got drawn into spec and that I think if you want to use easy building stuff you can do that as well.
Erik Schnetter: suspect is probably more used in the big access scale hbc computing centers but that doesn't mean it's easy to use and.
Erik Schnetter: Stop share my screen now.
Beyhan: Thank you so much for for Nice and informative poke anybody else.
Beyhan: Is there anyone wants to ask something.
Steven R Brandt: i'll just volunteer that if you start using spec and if you have some questions I might be able to help you because i've i've run across a few bumps in the road and and work through them.
Erik Schnetter: Thanks for not speaking up too much during the tutorial.
Erik Schnetter: These letting me advertise to me.
Erik Schnetter: yeah so originally event I.
Erik Schnetter: volunteered to kind of tag team that presentation, apparently, it was me giving the presentation, but that doesn't mean that Steve knows any less about me.
Steven R Brandt: I don't remember volunteering to be part of a tag team, but okay.
Steven R Brandt: Oh.
Steven R Brandt: No i'm sure Eric knows more about it but i've run into a few of my own special issues or things that I want to do, you can, for example, you can force back to check out a particular package in a particular directory if you're trying to debug it and something like that.
Erik Schnetter: i've never used that that I should I should know to do that.
Steven R Brandt: yeah i've got a little script to help me remember how to do it so.
Steven R Brandt: Although you get a sinking feeling once you start doing something like that right.
Erik Schnetter: I want to take this opportunity to advertise modern languages that package managers built in.
Erik Schnetter: For example, in Julia if you want to install html5 type ad html5 to add mpi type ad mpi I think to to get the bad idea, and it is really much easier because it's a binary.
Erik Schnetter: condo and Python to do the same rust is its own package manager cargo and it's really good and so on and so forth, so these languages fortran C c++ are really way behind the curve approximately five decades, I assume, and I really hope that tools like spec catch on and improve the ecosystem.
Beyhan: No one else would like to share anything anything than this, I would like to thank you, everyone for joining.
Beyhan: And pulled into meetings.
Beyhan: And when I stay.
Roland Haas: replace the color factors with.