Jutta Steiner: Secure Smart Contract Development

[Music] really excited to be here with you today unfortunately in this remote capacity and it's a bit odd past midnight here in an empty office that's been empty now for two and half weeks already but we do what we can so let's get started so I have been involved with crypto and security now for more than seven years I think jointly theorem team in 2014 as the chief security back then this was all a pretty new field like I joined to help sort out the security audit prior to launch and then also ensure operational security but there wasn't much at the time no audit firms that were dedicated to this I stayed on throughout a theorems launch and was around for when the Dow heck happened although that was already after we founded parity before I joined etherium I worked as a consultant for McKinsey in Berlin I was involved with various IT transformation projects but also IT security and larger corporates organizations so I have seen that angle as well either although my mathematician I'm not trained in cryptography I would say I did study relevant stuff so I was involved with understanding complex systems mathematical physics how structure forms but also how chaos forms and how small parameter changes can actually to severely different and dramatic changes in outcome which i think is very relevant in this space I was always intrigued with how maths can solve problems like mainly from university I knew how it can solve will help solve problems and the sciences so I was very intrigued when I came across blockchain and finally saw something where maths can help solving societal problems and not just analyze them which I was used to at the SS from Statistics at uni so that was sort of I got in through seeing that this technology could help with a lot of privacy issues we've had online and so I've always been very intrigued from the security standpoint with that technology so yeah right now I'm the CEO of parity technologies london-based company big team in Berlin I co-founded that together with Gavin wood in 2015 I guess five years old gavel was the CTO of the theorem foundation at the time and had developed the ether and virtual machine I would guess the first instance sort of smart contracts in the context of blockchains had taken that early idea of a blockchain scripting language that Vitalik had developed and sort of very well that you added into this comprehensive virtual machine framework and then the smart contracts a parity we do open source blockchain based software tools to enable other projects and developers to deliver on decentralization we've had a lot of learnings security-related ourselves and i want to share a bunch of them with you in our company I guess we have our very own view on security and last year and summer we sat down and and broke down a bunch of principles that sort of are important in our organization and when you compare that to the other more prominent startups and I guess we aim to move with enough caution and intention to not break things and having an undo is always a good thing so with that in mind I guess many of the things that you might hear today might sound like a bit like dry remarks but in hindsight a lot of the things in security seem to be always easy and super obvious however many of the things have been very hard and painful learnings and none of us should repeat them unnecessarily I think so that's my main goal to help you not do the same mistakes that people have done before and like a normal life like these wisdoms often sound trite but in the end they do carry a lot of wisdom and and truth and so it's worth exploring them with curiosity finally I think we've been always deep believe as an open source and security is one obvious dimension about open source I think wins against the closed source models in the past the this future decentralized web is not built by a single company and we've seen in many instances like the dow hack for example how projects came together and I collectively tried to solve issues when they came up so I think we've come a long way since the beginning when Bitcoin was invented Nakamoto consensus security has always been very much at the core of blockchain as a whole so nobody I think really would have expected or maybe a few people not definitely not the majority that's something like consensus Nakamoto consensus could work at all in fact I think there's a bit of a thing where Satoshi even urged WikiLeaks to not use Bitcoin to receive contributions because they were fearing it could harm and lead to irreparable irreparable harm - in case things break so but then things moved on so Bitcoin worked until hacks but mainly on the fringes few security issues directly inherent to the brockton network but then with the advent of a theorem a whole new attack surface came and that was mainly due to that virtual machine that allowed for a lot of things to go wrong the Dow being one of the most prominent things I guess in the space now where we today a lot of things have happened like the the base layer the lowest level things I've matured where we are trying to include standards from other industries or other places like webassembly but at the same time then app ecosystem is also maturing and it's becoming increasingly complex and and why we do have more tooling compartmentalization standardization things will probably continue to go wrong in the future so I want to share some learnings in particular that are relevant for smart contract development in this presentation but then also give a bit of an outlook of where I think things are heading and what people I might want to consider when they when they start a project in this face what are the security considerations you should make but first of all I think it's really important to keep in mind security is not just code this is a definition of NIST the National Institutes of Standards and Technology of an IT vulnerability it's a weakness in an information system the system security procedures internal controls or the implementation that could be exploited or triggered by threats or so it's it's various things it's the people in the organization it's the operational procedures it's the lifecycle management of the application but there's also a lot of things that might be totally out of your control it's it's the devices that people use and and and witnesses that could come in from that side but it's also important to keep in mind exploits don't have to be all deliberate and some of the full impact of an exploit might not have been intentionally at all so it's like a lot of things to consider and as Bruce Schneier who many of you probably know said he's one of the forefathers of software security cryptography intent privacy security is a process it a product can never be secure like they'll always be weaknesses things move on and you have to ensure that the entire system can be as secure as possible and that requires pulling levers and all sorts of level so Friday night enough of preaching of like principles high level um that's five into the smart contract bit of this presentation so when I use the term smart contract I mainly primarily refer to smart contracts as they appear on any theorem that's the space I'm mostly familiar with but let's take a step back what can go wrong with code just in general when you look at cold what are the things that the typical could go wrong in order to learn from that and how that applies to smart contracts many of you are probably aware my memory safety is always an issue not just from a security perspective if your web developer you're probably well aware of the disastrous effects of malicious input to your code and having to properly check for that systems administrators probably seen how accidental access controls can be very detrimental to what you're doing one of the things we are we're in particular trying to overcome with with Roc showing like the dependencies that come from very fundamental design decisions tossing attacks a very important issue on the web so yeah that's just just a few things just to give you a sense of what am I talking about when I say say things can go wrong with code one thing that's important to see is that all of these issues don't have like a single answer there's various things and various approaches we have to take in order to mitigate them at the core to me is the threat modeling so having a really thorough understanding of the potential threats the actors their motivations all the different vectors in how you can attack so that's that's a very fundamental thing you should always do and it's actually quite fun to do um when you start doing that and there's audits so reviews both internally and externally that you can do and which initially were quite difficult to come by with and the block sign domain but these days they're a bunch of specialized companies some specialize in smart contracts some do more wider work testing is important um obviously for Grizzlies reason both manually as well as fuzzing so random approaches for finding the issues with your code so how does that translate into smart contracts that's issues that can go wrong just in general with code so first of all like the issues we find in smart contracts and there's for example quite prominent security firm trade of bids we also work with them on polka dot who have done analyses of that and I think the reading material had some reference so they've looked at it and in principle you find the same issues with smart contracts that you find with no mokou not so surprisingly in the end it's the same monkey sitting in front of the keyboard but that that doesn't necessarily mean that it's in the end the same impact that it has right so a lot of the findings almost 50% were impossible to imagine detecting with with testing and that's really problematic because when you once deploy smart contract code it's hard it's hard to change it unless you've you've implemented measures beforehand they let you upgrade your code or make changes in whatever reasonable way there's really no way to what's really hard to to mitigate issues that come up also when you look at what's been written about you could get the impression that some of what you read about shouldn't necessarily mean it's it's the relevant code of the relevant issues for your application like other things might be more important because your your system displays other complexities but all in all I think the big learning from from all the issues that have come up is smart contract development is the opposite from agile it's the opposite from how software development happens today it's very costly the only because the only way to get additional insurances is basically by contracting external consultants in the form of audits and and they are really crusty which makes it a tough thing to do you want to probably apply a very thorough approach for writing your formal specifications and whatnot these are things you might want to take into account so I would say as sad as it is and even though tools have been developed smart contract development is far from easy and far from secure at this stage unfortunately still nonetheless there's things we can do parity we have developed like a 14-point checklist at some point that goes beyond just the development it's a very comprehensive list starts with how you organize your github and repos how you do the deployment the lifecycle management and then in the end of course also like how you deal with code quality and the inter production of the code so maybe just a few examples so what you want to do is you really want to compartmentalize what you do and separate it from everything else that's happening in your organization and you want to create a different github repo to ensure you have the appropriate access control in place you want to have separate repos for each contract also again to manage that a lot more a lot more granularly embed all dependencies to ensure you are always in control of what you're doing on the deployment side a lot can be learned from from best practices in other industries make sure that you that you always have a thorough understanding what is the actual life contract that you have deployed make sure that the readme is really comprehensive and gives you the relevant information that you need when things go wrong like where's actually the deployment address things that might seem travel really really aren't review when things go wrong and ant around a lot of pressure code quality you one might want to try and use a different language when you write the tests from what the code is actually written in to ensure you're not getting caught by language quirks and the review even if you don't engage auditors for whatever reason at least have the record reviews with a new organization so that's a few things on the on the smart contacts beyond that I think there's some interesting observations of how the entire ecosystem has evolved as an interdependent open ecosystem and things we've observed that our security relevant so constrained by the design space and performance limitations a lot of projects have started to actually build their own chains I think one of the first examples although a theorem was maturing back then was protocol labs with five coin and they started to build own chain but there weren't want more examples more recently and sort of a concerning thing when there is the old adage from crypto don't roll your own crypto rolling you all chain is maybe he's something you don't want to do because it's really difficult I mean we're we're a parity where we are focusing on that layer like building block chains we've build an e theorem implementation Bitcoin Z cash all sorts of things these days polka dot and there's a lot that you can apply learn from when for when you're doing a new chain and we've and we've encapsulated and we have consolidated that also in a framework that people can use but but if you start from scratch not only is that costly it's also from a security perspective not a good idea other things that play a role in that context is because of the limitations on the scalability side when you when people start to launch new chains in order to be able to deploy what they want to deploy at some point chains will start to compete for security so basically the mining resources validated resources being a scarce resource chains will eventually compete and we're seeing that like we're seeing such an era approaching with the 51 percent attacks that some of you might be familiar with on eath classic last year and other things this is becoming more and more problematic as things progress on the other hand applications are getting more complex and in particular intertwined with the platform as well we've seen that the for the first time I think or that became clear in the context of the Dow as well we're in the end the reason why that I'll get fixed and for those of you maybe not be too familiar with that that was a very the very first incident of a decentralized autonomous organization deployed with the help of smart contracts that dramatically failed there was an exploit and a malicious attack I was able to basically seize or would have been able to seize the funds in the Dow if they hadn't been affixed and since there was so much ether deployed in the towel around 10% the eventual decision was basically through governance move to to fix the issue in the smart contract through a hard walk so that was sort of the first time that it became clear there is an issue when the economics of the application get too intertwined with the fundamental security of the platform and I think very similarly for those of you have followed that in the defy space where is used as collateral that creates problems as well there isn't enough compartmentalization I would say what can we do I think that's not an easy question security isn't easy but I do think there's some stuff that we that we can do and that is happening quite interestingly as the technology is maturing let's let's look at scaling in particular so a very straightforward thing to do when you think about a system or technology that's limited in scalability is you would employ some way of parallelizing and whatever computations have to happen in order to improve the scaling very straightforward approach however depending on how you do this this actually works better was a very naive approach ties back to Bitcoin already with the site chains is to basically just run copies of the chain and try to paralyze them and have bridges between those chains that allow for communication across the chains however the issue with this is if you do this in a very simple simplistic way is that the security of the original ecosystem basically gets divided across the different chains and so that is basically what's in the end the problem because all these chains are much easier to attack crypto economically because of the security being divided across the chains but also you're running into a sort of a weakest link problem if you think of interoperability between the chains and that's really something we want because otherwise we end up with the same world gardens we already have at the moment but if the security is divided in this way you fundamentally rely on the weakest link in this sort of chain of chains because you have to rely on the security assurance that you get from those chains and and that gives you only a very weak interoperability that might in the end lead to severe problems when you build a complex application that tries to compose different things from different chains you don't have strong guarantees a much better approach is to try and share the security amongst the different chains like sounds obvious but it's actually quite difficult to engineer it's very closely related to the early shouting ideas for theorem so in such a multi-chain system all the chains share the common mining or in more modern systems proof of stake resources the great thing about this is that all also when it comes to the interoperability between the chains you get much stronger guarantees that you can actually rely on the messages transactions do you receive from the different chains and thus get strong strong assurances wrong around what you're hearing from the other chains now we can also drive this a level further and I explain later in the following slides what this all has to do with security apart from the sort of first level security when we drive this a little bit further an interesting thing to then come up with is to say well if we have such a framework why not allow for heterogeneous chains so in contrast to the theorem charting ideas where you have sort of the same copy of a theorem same virtual machine you allow for different application layer customization so different virtual machines or better set runtimes that allow for customization for your specific use case think of one of the chains as being a payment chain for example or one of them being a I don't know a supply chain specific chain or whatnot the the big advantage is not only do you gain much better much better economics because it's customized to use case but also does it it does it reduce the attack surface on your application and on the chain so again compartmentalization is one of the principles that is always has always been important in security and it's about like finding out how to do that in the right way in the context of blockchain so that's been a like those principles that I did I try to explain like a lot of this has been behind the thinking for how we have designed polka dot substrate and how we think about these days about what what platforms need to actually provide in order to allow for much more secure or use it so much more secure deployment or application for applications and that's not just relevant because it reduces the cost but in particular for security reasons like very similar to to security in ite as a whole whether isn't a single system or single mitigation strategy that helps you to close the holes also here like you need various layers to various layers of of tools that all have their own weaknesses their own holes but as a whole hopefully we can cover everything and create systems that are really difficult to penetrate so that customizable runtime model I think is a very powerful one because it very much so mitigates the the attack surface on the virtual machine level and like the dial heck we've seen the multi seek hack on our site for four for parity a few years ago many of those really really big attacks that have happened or issues on a theorem in particular I would argue have been related to the virtual machine being way too complex and and way too powerful as a tool in some ways way too powerful in some ways therefore way to expose and therefore actually way too weak so that's I think a pickup at unity for making things a lot more secure other things that you can do when you can leverage such a customizable framework is it's much easier for teams and projects to resort to standards like was a map assembly and much safer languages like rust for the writing of smart contracts because you're operating in a framework and not on a but the one fort of one-size-fits-all and still like very glued together solution so another thing I think that's also a lot underestimated at the moment is the issue of governance [Music] and how the open-source ecosystem the core team feeds into this by choosing a platform you make yourself dependent on them to a certain extent depending on whether they exist governance structures and what they look like and the problem is in the end on chain governance might be the ultimate measure in case of a big failure we've seen that we see networked with the dowel and through that fix and and having a strong approach on that and as I think or going forward and we'll say if safe people when things go wrong so still like a ton to figure out we've come a long way there is a lot that we can learn from other industries I think and with etherium we've we've done that as well when we started one of the things that we did in contrast to to Bitcoin when we started Bitcoin you might be in familiar with the fact that there is no there's just the reference implementation there is no formal specification or at least there wasn't originally with a theorem because we knew we would build something a lot more complex it seemed to make sense to us to rather like an aerospace start from a formal specification and then develop fully independently various implementations to achieve redundancy and remove the reliance on a single implementation because there's formal specification is a lot easier to check then then trying to understand what a reference implementation really does medicine again medical professions I think there are a lot of good procedures to be learned checklists one of them I think they've created a lot more safety and and a lot of the things we see is seem trivial but they should be written down in in a checklist in this in some format that can be easily accessed gone through when things go wrong we we used to run we always had frameworks in place for how to do and now like there's there's all sorts of things we're clear checklist can really save you when when you're under pressure in many ways I would say blockchains or like said say the smart contracts that we deploy a lot more similar to hardware than software once deployed once shipped very difficult to recall so might want to have a look at things that people do there when you are active in this space it's good to have the right procedures in the organization there's various things you can do I think in general you want to establish a really strong security culture in your organization's but one of the specific things you want to make sure you have is crisis communication because when things go wrong you don't want to scramble and try to find a firm that can help you talk to the press if that ever becomes necessarily and then again I think developing developing your code in in an open-source fashion has tremendous benefits it is I think one of the key things that that kept the theorem alive in the early day is the fact that also in that ecosystem projects came together and and helped each other and it's the same thing we're seeing now now for pokhara what was also powerful to build ecosystems but it's at least equally if not more required in order to create secure systems so key takeaway from this I mean I hope you did take away something from the from the practical things that I talked about if anything I provide you with some crypto adages so he can sound like an expert I think yeah one of the key things to keep in mind security is really much more than code and everybody in the organization should feel comfortable to speak up when they see an issue arising that's really the the best way to ensure that that that you are prepared things go wrong ideally that things don't even even go wrong like a strong security mindset in your organization is important smart contracts at least for now because of the lack of tooling on secure or at least say it's not easy to develop them in a way that you should rely on them it's not agile cost a lot and and more issues will probably happen until we have those tools be careful when you roll your own blockchain or better don't do it like like rolling your own crypto you don't want to like not rolling your own crypto you don't want to roll your own blockchain that's that's a big big undertaking I think overall it's it's important to stay humble when you're active in this in this field learn from other industries learn wherever you can there's a lot to be learned also from web 2 and stuff that went wrong and then I think lastly security is really really hard things will go wrong in the future and I think it's it's quite crucial that we recognize that we in this together and that we should learn as fast as we can from from each other to to build a better better web thank you great thanks you too that was great so yeah and good about the virtual around it was a visual round of applause great okay so so great we got some questions in the Q&A Channel and I'm gonna just sort of read them out to you and again thank you so much for staying up super late on Friday night tech to take these questions so the first one is from Paul and this question is what audits be more difficult for a future involving multiple chains on polka-dot when compared with a similar feature on aetherium since you need to consider the characteristics of every different VM that's involved I guess it I guess it very much depends I think it's easier to analyze in its in its components because on that level things are a lot easier to understand like if you have a customizable run it's a lot easier when it comes to the interdependencies I think and and and like sort of what can go wrong on a systems level as emergent effects I think you would end up with very similar things because at least I wouldn't I wouldn't assume it's it's more difficult over other assume it's it's at least it's it shouldn't be more difficult than anything because the underlying components are easier to understand so that's that's my since okay cool I see Paul nodding so I think we got it okay cool so next one is from Eddie on a 6 + Z team he's asking can you say more about how security guarantees can be made between shards of the same chain what example what are examples of projects that do this and some differences between these efforts security assurances I mean the only examples really that that come to my mind that that do have the strong interoperability eye serum and poker or like all the other attempts with with with doing and hopefully the TP that with the cosmos or POA like they all leave you just so the weak interoperability because they don't overcome they don't overcome the issue that you divide the security in the end and there's a big difference between sharing and dividing on that level that's that's really the only examples that that I can see ok cool next one is from Stefan Stefan what attack vectors open up by adding support for walls and smart contracts instead of a smart contract specific virtual machine um I would actually say take it from a different angle I think well yeah I mean it's a good I hear your question it's a you're asking about what what comes up but I think when it comes on a higher level it's it's the better approach because you're taking something that has a lot more a lot more eyes on it and then something that that is fairly limited in it in its audience that said I mean it's not that you you couldn't and there's probably interesting approaches for developing a secures my contract language but they say with solidity them that one of the primary goals was to make it very easy for people to develop that's why it's like for us a lot of ideas from JavaScript and not like security was in people's minds but people didn't know how they would be used it took ages for having a library approach in the language and whatnot I think it's just it's more about like is it really worth reinventing the wheel or what's the trade-off and and doesn't it make more sense to have these standards because they easier integrate he don't he don't create additional complexities through the integration people are familiar all these things that's why I would strongly argue for standards as compared to reinventing and making it specialized okay great next question is from Jeff and he asks how should projects think about making contracts upgradeable as a way to address unforeseen security issues I mean what you probably don't want or maybe you want it at least in the initial stages is a kill switch because then you have a centralized control again but there are ways of thinking about like how certain portions of the contract could be upgraded like having having places having things in place for that maybe then a multi-sig around that so there isn't a single controller that's ideas you could have when thinking about how to fix issues in hindsight decide great okay next one from aria what are some social attack vectors you're thinking about with regards to the non-technical side of dowels and things like on chain governance how has misinformation thought about from a security perspective and what's to mitigate that I mean the game theory of these things isn't easy that said some of the game theory it's not really clear how some of that plays out in the the real world I think that's in general something to be aware of in crypto and that started with Bitcoin already like not many people people would have thought that it that it that it does work but then it's the real world that makes or breaks and only then you become aware of like what the actual assumptions or modeling threat models the really pre-pro a meter should be so yeah I mean that's a hard question I think with with governance in particular we'll have to we'd have to see what things work and and won't as we're bringing in humans again and so they have really had to model in their behavior and I think there's more more unexpected things than sort of formal weaknesses that will come come to be problematic be that like voting participation and actually planning for that which is one of the things that we're doing like there's no voting participation you need a bigger quorum for things a small we got quorum for things to go through so like I think it's it's those things that are related to some of the practical considerations that will become important yeah one I actually have a follow-up to that which is I guess how do you how do you think about the the role of sort of traditional management structures in the context of governance versus like some of the new things you can you can do with smart contracts like quadratic voting for example like I don't believe all of what we've done so far should be thrown away like I do think like a lot of the things that we've developed in the past as governance mechanism mechanisms are like good governance in particular checks and balances and whatnot like to really think hard about like how how to do that to me it's like the power lies really in making things more liquid and being able to play with voting mechanisms that we haven't they have been so far really difficult to be implemented and used because of the cost involved and whatnot like to me um if we can do it right like it's a big field for experimentation I'm trying out what what things actually work because on paper that they may they might they might look like they were and then they don't or the other way round because we haven't considered I'm a lot of the issues but always do it in a way where you have a noun do well you can't go back I didn't okay great so next one from ghee are guys sorry how would you how would you try to defend against supposedly trusted contributors introducing obscure flaws for the sake of attacking a network for example of a core dev introduced a vulnerability with the intent of exploiting it for profit mm-hmm I mean I guess even with open sores like shorting is always something that people can do when they introduce something but I do think open source already mitigates against a lot of these risks I mean in the organization again when when it's an organization that does this it's relatively easy to implement relatively strict review processes to still ensure there is there is redundancy not the reliance on on a single on a single contributor okay next one is from Marc and he says advice I mean again tying back to where I started like some of the threats that your system might be exposed to might not even be deliberate right like you want to do all these things even because people just make mistakes like regardless of what the demolitions I mean with their malicious they might might make like sort of sophisticated mistakes deliberately and and they're hard to come by with a but like a lot of the things you want to have in place in any case if you're dealing with such such core essential services because because things can just can just go wrong accidentally right so yeah it's two-tier point like you need to you need to have multiple processes in place so if an owner Bodhi gets introduced in that way hopefully the process catches it right yeah and reviewed it I mean take take all sorts of approaches for for coming up with that like and be and be creative for not falling into group thing like when you when you sort of request maybe feedback from the organization on what they think vulnerabilities are in your system then then ask for anonymous feedback so people don't fall into the groupthink way like there's a lot of smart things you can do that help you deal also with the weaknesses in terms of psychology of people right okay so back to Mark's question so you advise not to roll your own chain and there are options for for not doing so so there's polka dot substrate and and cosmos sdk and his question is which projects have been successful using these and which have done well rolling their own chain mm-hmm so projects are starting to deploy I mean it's still early days a lot of projects will come out with their chains over there but their parents for baccarat once one spoke about his life and it might be easiest if I send some examples if that's helpful also happy there particular things you're interested in that I can send around yeah yeah that'd be great I guess maybe you know I think we're still sort of the dawn of this era of building your own chain with and sort of SDK like the ones that we mentioned yeah be cool to see some of their own examples of that and if yeah if you have any that are top of mind would love to share in the side cool so the next one is from Stephane again so how do we align incentives between those who write the code and review the code with those who actually lose from hacks well I guess that depends I mean - so when it comes to the days I think that varies about various aspects you like there's a dealing with dealing with the malicious malicious developer that wants to short maybe that's a that's I think a specific case I think there's there's interesting ideas around no sorry bounty programs more like also Treasury dolls that could help finance the development of open source just in general that's one of the things that we that we are also doing with with polka dot like think about like how long-term in general your ecosystem is supposed to run in particular and open source I think it's not so although like security is again like a key aspect in general you want to make sure that you incentivize the contribution in the right way so that's that's one of the things otherwise I think it heavily depends on on the application like if there's a if you are if this is now about open source in general or if this is about like how do you do that in your India in your organization I think that's that's two very different issues right there actually that gets to another question that I had which is this is from me personally its its how do you think about hiring for security in the organization like who are the key hires that you need to make and how do you build an organization around it subsets yeah you can ensure the processes are being carried out across your record mm-hmm I think that I think it again like that depends on what you're doing I mean you want to have experts for for like what you're what you're delivering in the at the core like you certainly want to to hire people for that but if that's like very cutting edge like say you're using substrate to build I don't know a zero knowledge proof chain or that includes your knowledge capacities like you'll have a hard time finding X but then you want people who really can think outside of the box who are like really happy to to to think first principles sit down you will need to work with all sorts of people across the the system like with the early audits back in here we were doing the same for polka dot like we work with academics that look at certain theoretical aspects of what we're doing then there's the record auditors that depending on their specialities look at relatively straightforward things where they can do like standard approaches like fuzzing that they know from other systems but then for for some stuff you really need somebody who just sits down and things really really hard about things you're doing and and it depends it's good to have I mean if you can find someone but they're really hard to come by with like experienced people who've seen things go wrong in this ecosystem that's that's always always very useful but I wouldn't underestimate also it's not enough to have you won't make some things secure just because you have one expert right if you're you're building a culture of like move fast and break things then even that person can't help you with issues that will come up yeah so do you maybe do you think there's some something to be gained maybe from hiring sort of security officers chief security officers from other more established their software companies or some of the other domains that you mentioned in your talk I mean probably I mean like we I think we are wearing a relatively special situation where we're developing like technology at the cutting edge I mean even there I think like well we we did look at at at aerospace which has similar issues like which just it's something like when you build a new a plan like this is a totally new thing and there's a lot of a lot of considerations you need to make there's a lot of rules you have to comply with like all these things and and and and learning from that I think makes makes total sense some of that is not easily accessible like no it's not because open-source is not so called to those industries you don't necessarily find a lot of the information but it's certainly worth talking to people from these domains and learning from them cool okay so moving on to the next one do you have any examples of exploits that weren't at the code level yeah sorry any any examples of exploits that weren't at the level that come to mind from from your experience with existing blockchains any exploits that won at the court level like I'm not so I guess they made that ties back to my statement that I made like security is not just code like like issues originate often in in in code I mean typically they do that's what we're building but then the the mitigation and and how you how you deal with them like that goes goes far beyond the code like even if there is an issue in your code like you want to make sure you have the procedures in place to catch those issues on various levels like that's the main point point like in the end like we develop a code so some things that break are typically on that level I mean apart from like all the social engineering attacks when people try to steal crypto right like that that's a bit of a special a special set of of issues that have come up in blockchain yeah yeah but yeah as you mentioned in your talk like you know 51% attacks where miners of one network are attacking another that's that's that's happened many times and that's a that's an example it's not at the code level it's at the sort of interdependent level of the ecosystem I mean yeah sure like those vulnerabilities yeah exactly so there's the there's there's all the things that come in because we're dealing with crypto economic systems here like 51% attacks everything we've seen in defy in the last few weeks like the way how we how we expose how we get exposure because of market dependencies and whatnot like that's a huge thing to keep in mind and I think it's not very well understood it's one of the things that we also try to address with the way now we designed i'm substrate info corrode in particular so that you can have independent economics of the application totally independent of the economics of the platform so one thing any theorem is you price you price all the transactions in ether in the end because this is how the virtual machine works but if you design things a bit differently which we do in fact like that's that's not that's not required like you can have independent economics of the runtime and so first of all you don't expose your users to to the underlying cryptographic chocolate token in that way but then also from a security and and and and and capex and OPEX perspective like that's a way better way of dealing with the deployment of the application ok yeah that's great and that actually that gets right into a question from mark which is do you think it makes sense to divide the security in a case where not all the transactions need the same amount of security so and if security cannot be divided then low value transactions may have to overpay for security and compete against high-value transactions totally totally um I mean I'm not sure I can come up with a good example but a different way of thinking about this of where like sort of similar considerations play a role if you think of you wanted to build I don't know a platform for for gaming people that acquire assets in a game over time and that that's like on the chain I'm shade because people really want to own this like initially it's probably fine if such assets live in sort of a private chain setting because they're not worth much but then once such an asset becomes worth a lot like there should be an easy transition for that asset to exist like on a public chain like I think that's that's very similar things like their security in the end is always a trade off like really strong security is really expensive and always will be and sometimes it's also to practically not feasible so yeah totally there's there's probably ways to think about it in that way but be keen to hear any any further thoughts you have on that end yeah I mean it's it's a it's a complicated I I don't have deep thoughts on this but it's a very it's a very complicated subject right and and I think it's or as it gets into as we talked about sort of the interdependencies between chains if you have a chain of like high throughput that very value transactions that could be fine for that community but also easy to attack if someone was motivated to do it yeah yeah and I think there's a whole range I mean because we're just starting to see sort of more real world applications being deployed or like actually use cases being deployed like that we have have not understood at all and where they're where we will have to deal with new issues coming up I think it's also an interesting moment as I mean like the hype has gone down but I think also more and more people to recognize the the actual relevance of blockchain like I think it's totally worth making efforts at educating sort of the more the more original cryptography crypto community I think many of them still think like blockchain is like oh maybe not many but but I think there's there's some skepticism and and talking to them because they have done a lot of good work where we we all can learn from ok so next one is from Eddy and Jeff what kind of tool is formal verification crypto economic simulation for example do we need in order to make smart contracts more secure in the long term and how should projects think about the trade-off between the time and the cost of doing these verifications versus shipping hmm then that's a that's a that's a really really hot one I mean it depends on the skill level you have like in the organization but then again even that like won't necessarily save you like there's there's a lot of considerations um I don't think that shipping I mean yes comp as much as you can compartmentalize and and and like minimize the the value at risk like all these things like like take all these these principles and and try to think how you can mitigate the the the worst-case impact outcome and and like yeah and I'm not sure there's like a again there is like no single answer there's like all sorts of things you need you need to consider and I have a hard time making a strong recommendation as I don't know what what the considerations are you're taking yeah so it's Swiss cheese right like many layers of say cheese okay course so I mean in the end life still I mean as much as you can like I mean do try to think about like how could you how could you mitigate if if certain things can come up like definitely um even if you decide to ship fast make sure you're prepared for things to go wrong like because if things go wrong like they can go awfully wrong and you do want to prepare for that for that case so um yeah another another one from Ryan is what security considerations should be kept in mind when evaluating relatively newer block chains to build on the way how you can how you can in that framework actually understand the attack surface and and how you can the threat modeling and how you can mitigate against those like what what are the tools that the the applique that the framework actually gives you in order to minimize the attack surface that you have I think that's that's a that's an important thing to keep in mind when you do this yeah okay cool so I know it's super late it's past 1:00 a.m. I think for you so I think we can we can pause it there and again thank you you so much for the presentation and for answering all the questions [Music] you