Talking SAST, AppSec Tools & False Positives with Florin Coada | AppSecEngineer Podcast

In this episode of the AppSecEngineer Podcast, Abhay Bhargav is in conversation with Florin Coada, Product Manager at HCL AppScan.

He talks about why he doesn’t think false positives are always a bad thing, how his background as a programmer has changed how he looks at application security, and even how he and his team deal with the ever-increasing numbers of programming languages used to build tools. In particular, Florin talks about the relationship between engineers and the security tools they use, particularly Static Application Security Testing (SAST) tools, and even his vision for an ideal future in which we won’t need security measures to keep our data safe.

Talking SAST, AppSec Tools & False Positives with Florin Coada | AppSecEngineer Podcast

Abhay - Hi everyone. Welcome to another episode of AppSecEngineer. I'm really excited about this episode because I am talking to somebody in the Application Security Tooling Industry. So one of the things that all of us use, especially in Application Security is we use tools. We use Static Analysis Tools. We use Dynamic Analysis Tools. We use Source Composition Analysis Tools. And of course, today we are using Cloud Security Tools, Container Security Tools, and all kinds of tools. So one of the things that I always like to do is, you know , talk to people who are implementing it because heaven knows it's a tough thing to do. It also it's constantly changing.

The landscape is changing all the time. So I always wonder how popular tools keep up with what they do and how they do certain things. So today we are going to get it straight from the horse's mouth. We're going to be talking to Florin Coada, who is a product manager at HCL AppScan. I started interacting with Florin about, I would say three months ago. And it was very interesting. We had a bunch of conversations on LinkedIn and stuff and very interesting interacting with them. So I thought I'd have him as a guest. And see what he has to say about the state of Application Security and Application Security Engineering today. So welcome Florin to this.

So welcome Florin to this episode. Thanks so much for being here and thanks for spending the time with us on this episode of AppSecEngineer. 

Florin - Hey, thank you very much for having me here. I'm always happy to talk to people at this time in 2020, it's quite difficult to have face-to-face time. Is there anything else I can do? I'll do it. I will get on Webex. I'll get on a podcast. I will go the extra step to get here.

Abhay - Absolutely,  yeah. So, so nice to see that, yeah, so, you know, we set this up what a month ago. I think about a month ago it was, yes. Right, right. So, yeah, Florin and I like I said, we've been chatting about different things.

He did send me some recommendation for some tools that they have built, open source tools that AppScan has built, which was very interesting to me because I don't think this was a thing before , you know, the recent , you know, HCL, you know, the developments that are happened in your engineering. So. I think this is kind of new.

I had not heard of an open source tool from AppScan. So it was very interesting to see that you brought it up. It was quite good when I used it. It actually did what it was supposed to do. And I was very happy to see that.

Florin - So anyway, sorry I would call it free to use. We have an open source, so we still have the code to ourselves. We keep it to ourselves. But we did make it for use and it is something new for us. It's a community edition is what we call it. And just like you said, it's something new for us as well. We like to experiment. We want to try things. And we thought, Hey, we have something. We can actually make it easy  to use for people.

So why not? Absolutely. I'm glad to hear that. You find interesting stuff. I'd have to say something, you know, when you made the intro, you said something interesting. And I  wanted to start my whole , you know, my whole story with that. You say, you know, in AppSec we use tools and I think the important part that you said there is we use tools and this shows the fact that people should understand there is the human aspect to AppSec. Don't ever think that you can just buy a tool that will do everything for you. Humans are part of the process of development. They write a code, they need to understand the code. So as always, tools, whatever tools you choose, that's fine but make sure you build a process for the people involved, for developers involved.

And I just want to call it out because you did mention that. And I didn't want to lose that. 

Abhay - No, no, absolutely. You're absolutely right. Yeah. We do use tools. I mean, and while we try to automate some of it and we definitely look to automate a lot of it with some of the DevSecOps stuff that we're doing , you know, it's still human beings that are involved in the application development and application security process.

So very rightly said. Now with that, this is the first question I ask all my guests that come on the show. How did you get your start in security? So that's my first question. And I'll really .. Usually some people have very wildly interesting stories or sometimes not so interesting but I really want to know.

How did you get your start in security and how did that journey start for you?

Florin - It's a good question. How did I get started with security? I never wanted to be in security. [Laughter]  I don't know if that's something I should say or not but I always wanted to be a games developer or I thought I did for a long time.

I started my university around 19-years-old and I was like, okay, I'm going to become a games developer. I will write games. I will write code. I think I still have some of my university demos online. I need to, where I looked at things and building things for my university courses. But after that I got an interview with IBM for a position with them for what they call the Technical Sales Engineer, kinda like a Field architect.

I dunno what other people are calling it. And there's a few things being discussed and we discussed about mainframes, which funnily enough, I found interesting at a time and some other technologies but one of them was security and coming from Romania, it's a kind of.. kind of has a reputation for doing bad things when it comes to cybersecurity.

And I thought, you know what, security looks interesting. There are a few popular, popular movies out at the time with hackers and everything. And I thought, you know what? I'd like to give this a go. I think I might like it. I think I'll enjoy it. So let's do this. And after that my manager who back then hired me thought, what would be good for this guy? What would he like? So what he did is, he just did a Google like you do, you know, you just Google someone, you see if they have a LinkedIn or what they have, see if they have an experience or whatnot. And he came across my YouTube videos and some of the work I did there.

And he was like, he knows how to code. This is our new application security guy. And that's how I came into it. And it turns out I really liked that. I enjoyed a lot understanding how security works. Getting into the mindset of security. Getting into the mindset of, Hey, this can easily be abused to do something bad but I enjoyed it. Most of it was the technicalities and the fact that Application Securities tell development. A lot of people kind of see application security, hey, it's a branch of security, but the way I see it, it's still a lot of times, development, it's education. It forces you to learn a lot of new things. Sometimes, you can be doing a job for a while and you kind of get stuck but the application security have to learn about new things.

Since I started doing security, we all of a sudden started looking at Software Composition Analysis. Started looking at Cloud Security and now we're talking about Infrastructure as Code and even more Internet of things. And that's kind of how I came to be here. I've done the work as a technical sales engineer, whatever people call it for a few years. And then I thought, Hey, what's next for me? And it just happened that I was spending a lot of time with my product management team and talking a lot with them about things I'm seeing, things I would like to see in the product. And an opportunity came along, an opening came and they said, Hey, do you want to come in, maybe work with us?

And I said, yeah, let's try it. And I've done a year of my time kind of being the technical guy on the team, helping setting up different requirements for b products. And then I got my own piece, which is the Static Application Security Testing or SAST as most people know it in the market.

Abhay - All right. So now, I think one of the things that I've always, I mean, it's always fascinated me and also , you know, sometimes I wondered quite extensively is where do you see this thing going right?. So we have suddenly seen a sea change in the way applications are. You're a developer, so you'll probably have seen this during your own tenure developing stuff, writing stuff, and also being on the security side of the fence. I think you've been on both sides of the fence at different times. Where do you see this going, especially because within the last three years we have suddenly had this burst of , you know , the change in the way applications are delivered, right? So you have your micro services. You have your, the way things are going, it's much faster. Platforms are just , you know, mushrooming all over the place. Languages are coming up that we have not dealt with before. How do you, where do you see this going? Where do you , you know, where do you see this... it's going to end up with?

Florin - Hmm. It's, it's a difficult question. I would say on your previous comment, I probably haven't spent as much time in development as other people did, so while I was writing a lot of code and I still do it in my, in my time, and hopefully we'll talk about that later if I get some.... if I remember to mention it but I just spent a lot of time on this side of the... of the fence.

And I do like to spend a lot of time with my customers to understand where they are. I would split your question in two. You asked me where do I think it is going? I'll tell you where do I hope it's going? And I hope it's going away and it's weird. But sometimes we should think in the same way doctors do.

You know, what do they want to see where they want medicine to go in someday and I'm sure the answer would be I want it to go away. I don't want us to have the problems in the first place. I'm hoping we can educate everyone properly from university. I'm hoping we can spend more time making security as per the requirements.

I'm hoping a lot of that will happen, but where do I think realistically this will go. The analogy I like to use is similar to what happened to SIEM systems, the incident and event management systems, more or less. I see Application Security platforms or, or tools becoming more platforms than anything else where different development teams will use different tools.

They'll use their own choice and then kind of events pop up. And you as a security team, kind of do the assurance and tell them, Hey, if any of this happens, we want to fix it because we either send a threat where it is being exposed to. We understand what the problem is. We can easily set the guidelines for you, set the internal policies were... whatever we call them.

And then you just use the tool you want , whether you want to use Static Analysis, whether you want to use Dynamic Analysis or whether you want to use Interactive Analysis, some advanced Machine Learning, whatever you want to use. As soon as we see an issue, as soon as we see something happening as a security team will probably want to know about it but we'll rely on you, the developers, they will trust you to do the right thing and the right job . Where I want this to be is, I want us to be in a place where it's development teams will have someone that will know and will care about security enough to act as what we call a security champion. So security becomes a real part of DevOps and, of course, whenever I say security champions, people think, Oh! you know, security training, 20 years of hacking with it just needs to be someone who knows how to try to save SQL prepared statements. You know, it [Laughter] It doesn't have to be a lot more fancier than that or someone who knows where to find security resources. So the final place where I think this will end up is development teams with a person that's kind of responsible for running security, making it automated wherever they want to be and then security teams who can once in a while keep an eye on what you're doing and saying, yeah, you know, you're doing the right stuff. You're doing the right thing. As far as the technologies and the innovations that are probably going to happen in this space, it's always difficult to guess. I think everyone has some intuition and everyone has some favorites they would like to see coming up.

I think everyone is talking about fixed suggestions and auto remediation and a lot of these things, which, you know, sound interesting. And it's cool to hear about. But in the end, I think what security will become is hopefully another tick in the box. You're doing your regressions. You're doing your functional testing.

You're doing your unit testing. Security just becomes one of those. If you do it as a developer, hopefully it should be more of a unit test. Kind of like what we've been doing with the, with the tool you've been playing with a quick checks.. while, then you'll have some testing in the automation step or your build pipeline that will tell you, Hey, you know, someone's committing code or someone's building code that's slightly insecure, do we want to do anything about it. And finally, at the other end, you'll have security. They'll be able to go ahead and say, you know what, I'm just going to do a deep dive of this. I'll do a Pentest. I'll use the tools. I'll, cranking up to maximum, find everything I can and tell you about it. But all of this gets fed in the same platform. So when you find something at a different stage, you'll be able to inform everyone else in the chain about what's happening and how ...how you should respond to this in the future. I think a lot of the times, one of the biggest problems of these tools is not the tools themselves, but the cultural approach to keep them isolated from everyone else.

Like you told us security and that's it. But in reality, you want the two or three part of the process. And again, I'm kind of extending my answer here, but I'm hoping that this ends up in a place where the tools are part of the process and not the process. I think it's one of the biggest advice I have for people out there doing any type of application security process, make the tools part of the process not the process because if the tools are the process, then it will be hard for people to accept it and understand it.

And it will be hard to learn security. They'll just rely on the tools to do everything. And you know, a lot of the times you don't have to, you can do a lot of it well ahead of the... of the tools.

Abhay - One of the points that you brought up was pretty interesting about tools being part of the process and people using tools as the process, which, which a lot of them do. I know, I know from experience and I can quite agree with you that they do that a lot. However, the other side of the fence is there's also the fact that now coming to Static Analysis, right. Now speaking about your subject, your focus area, especially within, you know, HCL Appscan Static Analysis. Now one of the common issues that we see with Static Analysis, not, I mean, not... I don't want to name any specific product or anything, but the entire industry at least for the longest time, one of the big issues that we saw or a lot of people had with Static Analysis  was false positives, right. And , today, of course it is, it continues to be a problem in some quarters. It does not continue to be a problem for some. I'm not exactly sure the broad state of it but definitely some places it's still an issue that a lot of, you know, some of my customers have faced overtime. What are the things that , you know, you folks have been doing or what do you think about, first of all, false positives and how are they? What is it that you've been doing to reduce false positives or address this issue in the broader scope of your product. 

Florin - False positives is a conversation that I really like to have. I like to say there is no such thing as a false positive. There's just another question for me to answer.

One of the things that people confuse or not confuse but it's... it's unclear when dealing with Application Security Testing, there are different types.  My cat is assaulting me.  He's trying to get in and he is moving my camera. What happens is when you deal with Static Analysis, you're dealing with an application, a bit of a void.

It's a vacuum. It's just the application itself. I don't know where it's running. I don't know much about the context. Nowadays, I guess you might get some information about, hey, you know, it's kind of running in a Docker environment or it's running in a node.js environment but a lot of times you don't know who's interacting with it.

You don't know how you're interacting with it. So a lot of the things we're doing is we're asking questions with Static Analysis. We're asking questions for the code and we're saying, hey code, tell me your secrets. Tell me, are you communicating over HTTP? Are you writing to a database? And then we come back with a list of discretions.

And when we have this list of questions, some of them we'll be looking at will be saying, well, you know, while we’re quite confident that this is a real problem. It goes to high severity. We're confident about it but some of them will be posed to you as questions. So a lot of things. A lot of times people think about Static Analysis and saying, hey, it's given me 3000 issues. While in reality, what's giving you is maybe a hundred issues and 2,900 or however many left questions, is this a problem? Could this be a problem? Because Static Analysis, it's very theoretical in practice. As data comes from the user, it goes to a database. It's a non-prepared statement. I know it's a concatenated string, so it could be a SQL injection but it could also be a feature because you're just happened to be the UI to write to that database.

So I just want to let you do whatever crazy things you can in your mind. You know, the sky should be the limit. And that's kind of my first take on false positives. They are there, you know, they are there in a way but a lot of times I like to call them noise. It's things that are not interesting to you because in theory, you know, something could happen; in practice, something might not happen but it's good to ask those questions and be able to answer them. And I think the problem is people want to fix everything. They run the tools. And this is where it comes to the tool is the process. They put things through the thrill and they say, you know, I got too many findings, I cannot work with it. While the advice we have and we've been given for a while is, hey, look, talk to your security team, define what's a threat for you. Focus on some of the things you can easily fix. Make sure you're really good at those. Make sure you are the rock stars of SQL injection, Cross-Site Scripting, OS injection or whatever you think it's a threat to you. Talk to people outside, see what they're doing and get that baseline really good.

And then expand it slowly because in that way, you allow developers to learn at the same time as identifying things. And after a few iterations, you're going to have good confidence that, hey, you know, they're only looking at these things. They know how to fix them. And then you slightly expand the goal and you start looking for more and more things and that kind of helps you get through this process.

And that's where the tools become part of the process and not the process because you have people advising with each other . You'll have that security conscious person in your team that can go and say, you know, I'm looking at this and I think it's real. I think I can fix it. And finally, you'll be able to escalate that to security and security will be able to help you more.

Well, I see a lot of times it's developers, you have the tool, security teams, and then you get a lot of developers asking security, hey, what's happening with this? They cannot keep up with the numbers. And then, hey, they were saying too many findings, too many false positives. It becomes a problem, how can I deal with this?

And it's, it's a fair statement in their view, you know. I'd be upset if I would have to answer 3000 emails all of a sudden. But what happens with me in emails is, I get a few emails every day that I will respond. I'll answer them straight up because I know they're important and a lot of emails that go to low-priority emails, let's call them, which I'll answer in the end. But you know, maybe at the end of the week, once I'm done with really high-priority ones. And that helps me keep a steady flow going. And when I talk about Static Analysis and false positives, I like to make this analogy saying, if you start Static Analysis, when your project is done, where after 10 years of development, of course, you're going to find a lot of stuff. If you do not focus on something specific that you know to be a threat, then you are going to struggle to handle that. It's just trying to answer 10,000 emails after your holiday. No one can do it. So you just look at the ones with high importance, you answer those and everything else gets sent to the low-priority folder  people call it and deal with that. That's part of the story. That's... that's how I feel about false positives. They're bad but they're good as well. They should make you ask the question and should make you think a bit more about security. In terms of what we've been doing with false positives, we have been looking at different ways of handling them and we have been prioritizing things and prioritizing up, prioritizing down and to do this, we've been using some of our engineers. They have years of doing this, years of training doing it. And they kind of build a Machine Learning Algorithm that goes through them and says, you know, this one gets prioritized high.

The other one is not so interesting. It's isn't likely something will happen with this. So we're going to prioritize it low. And at the end of it, which just see with that. Say the results, I would say, hey, here's what's really interesting and important for you. We grouped them together to make sure they're easy to fix. A lot of times the same multiple issues can be fixed with one or two lines of code. And we present that back to allow developers to expedite the work.

Abhay - So there were two things that , you know , stood out in  that... in that statement you made about false positives, one is about questions. And I would like to think of Static Analysis as questions myself also because, especially with the kind of the newer tools that have coming out in this space, especially if you look at CodeQL or Semgrep, they are essentially questions . They allow you to query. They query stuff and they essentially, I mean, in a very literal sense, they are questions. The other point that you mentioned was noise, signal versus noise, which is again, you are, you know, to put it naively true positive versus false positive, right . Signal versus noise.

[00:21:26] I mean, naively..naively speaking. Now, one of the things ... what would you suggest to somebody regardless of tool, forget, you know, AppScan, forget this, forget whatever, any tool that they have. I'm not, I'm not going to focus on a tool but how do you, you work with, obviously, you work with customers, how do you suggest that they

first of all reduce noise and second deal with noise? What would you suggest? What, what, what is your take on that situation, especially when there is noise? 

Florin -  I would say first off, define a policy, establish what needs to be fixed ASAP and what to care about. A lot of the time, it'll be a case of you have a security team and we have some customers where those teams are great. I've worked with people where the security teams are really, you know, on-point, they know what they want. They know what they're after, nothing stops them. It's... stop that. You'd expect sometimes to stop. You don't expect that thereafter, but they know what they want. So always, always, always talk to your security teams and others and while you want to fix. The second part of it is don't take it personal when security says it's not noise. But do be ready to educate yourself to explain why it's not noise and why there's noise? So when security comes back and says, you have to fix all of this, you will want to be a bit educated.

Say why it's not noise. And I think that's where education comes into play. And you want to spend time with it. You want to learn a bit more about security and having a security champion and having a true DevSecOps approach will benefit you a lot. It's one other easy way to deal with noise. And the third one is look at your latest, last, Pentest and see which areas of concern the Pentester has raised with you. A lot of the Pentest I've seen and a lot of Pentesters I've been talking to can do that. They can do a bit of Threat Modeling for you as well. They'll be able to tell you, hey, look, you're dealing with databases. You have a lot of database queries throughout your application, that should be your number one concern. Number two, you're storing files to a disk, you are talking to the disc. You should be focusing on that. Work with someone from the outside if you're not sure, go outside and ask for a bit of help. Sometimes this will be quite easy to do if you have friends. I think it's easier for us, security because we kind of know a lot of security people and we can help each other out but just go outside. There's other conferences. There's a lot of people talking about this . Learn a bit of Threat Modeling. Once you know how to do a bit of Threat Modeling, you'll be able to define your own policy for your application. And all of a sudden you'll be able to start saying, I don't deal with... [pause]  I don't know what examples to pull out here from top of my head, but I don't write to the disk.

So everything I see says that, hey, you're, you're dealing with OS injection, I probably doing things well. It's probably that API that brought something to the disk, but it's actually no; it's disk to disk. I've done that in my Threat Modeling, no one touches the disk other than me. So if they have access to the machine, the application would be of interest to them.

So I'll just say not part of my policy, but enforcing, defining your own policies and doing a bit of Threat Modeling can help you a lot with this. And education is a big part of it. Whenever you feel stuck, I think there are a lot of education tools out there. There's a lot of webinars about it nowadays. There is a lot of YouTube content, you know. I've been watching you on YouTube, I think that's how we got in touch. I started doing something really cool on YouTube and I was like, hey, let me reach out and see what he can tell about this thing I'm building. Is it a great idea? Is it a bad idea? So look out for it,

education information. Learn to deal with the Threat Modeling. Ask for help from external places. Pentesters are a great resource. Always, you know, try and talk to them when the Pentest results come back and say, hey, which are the areas I should focus on and they can help you with that. And then knowing what you want to look for can help you with the noise. You can't make the tools less noisy because different people want different things. So you need to have quite a big range of things you look for because that's where you're asking questions. That's how you can find some time bomb, some back doors. But in reality, if you know what you really want to do or you're really focused on , you can just narrow the list and say, you know, I want to filter by these top three CVS and that's it, you know, those will give me everything I want and that will give me everything I want and it will allow me to stay focused in my job.

Abhay - So one point that you mentioned of course, is a pet project of mine, which is Threat Modeling. So Threat Modeling is something that I'm big on. In fact, my, my take on Threat Modeling is that Threat Modeling essentially is a playbook for your entire Application Security Program, right. So to do a good job of Threat Modeling, you are, you can do a good job of Static Analysis because you can ask your Static Analysis Tools to look for specific patterns. You can have your Pentesters look for specific patterns. You can have your developers look, I mean, fix specific kind of bugs and not introduce specific kind of bugs. The way I see Threat Modeling is that it kind of, the benefits of good Threat Modeling kind of work across the SDLC rather than just in the planning or design stage of the product.

So that's, yeah, that's a very important point as well. 

Florin - As you said...yeah, You know , different parts of the process should be interconnected, right? And sometimes, you can get the Threat Modeling at the end of a Pentest if you just ask nicely or, you know, it depends again on how big your project is.

Some projects will require some serious efforts to Threat Model but, you know, for whatever small fewer or for, you know, a beuro or something, they'll be able to tell you, focus on this. And, you're right, Threat modeling can help you a lot. Yeah. It doesn't have to be super complex. The way I like to present Threat Modeling is it's fairly basic.

Does data come from the user.  

Abhay - Yeah. 

Florin - And when I say user, it does have to be a human, can be a machine, whatever, don't trust it. Yeah. Then if you do anything with that data, you need to make sure you have a role to look at it and understand it. And that's the most basic way of doing it. 

Abhay - Yeah. Absolutely.

One of the things that constantly fascinates me, especially in the last, I would say four or five, I mean, five years or so. You've suddenly seen this burst of new languages, new platform, new frameworks. You have Rust. You have GO. You have Dart. Ahh man!!... you have Elixir. How do you guys. Absolutely, I keep the track of everything they create. Caml and all this stuff. How do you guys manage? How, how do you ... how do you folks actually, first of all, you know, build  rules for this many things. And how do you manage patterns, your anti-patterns? You're invariants. I mean, I'm sure that's not trivial. How do you guys actually manage it?

What is your take on that? 

It's a big task and it's difficult and it's becoming more difficult. There's a lot of tutorials on YouTube, how to build your own scripting language and you end up with a new scripting language that becomes very popular. My favorite one, I've seen recently, it's called pod.js. It's another JavaScript framework. I think it's one of the nightmares of everyone doing application security in SAST, it gets me into a blinking . It's maddening. But I think one of the things we noticed before I get into how we kind of handling, one thing we noticed with a lot of these new languages, the people building them are a lot of the times they're more security conscious than they used to.

I remember reading a bit about SQL injection when I started with this and how it came way back when, and, you know, Microsoft said it's a feature and not a bug. And that's how we kind of ended up with SQL injection. But a lot of these times, these days, the frameworks that are being built, the languages, they try to be secure by default and they allow you to do things but they run the security code on their own. They don't expect it to do sanitization for Cross-Site Scripting. You know, you can just print the user input and all of a sudden it works. They don't expect you to go ahead and sanitize it. They do this secure by default work. They do a great job of it and allow more frameworks for us. Nowadays, it's just a case of, hey, let's look how you turn that off and fix it and create a rule for it, which is quite nice. It's quite beneficial for us as security, security companies, to be able to look at why they're saying, hey, here's some security considerations. There's a lot of research being done, you know, as how do we handle this and how do we find stuff? There's other research being done. Like everyone else, we have our own internal research teams that kind of look out for stuff and look at how we can mess with some things, how we can make it, do bad things in the right way. But there are also a lot of research of people saying, hey, you know, there's a problem with this.

There's a problem with that. There's a lot of information coming from the developers or the vendors themselves. When they build a framework, they say, hey, here's how you do this securely because it is a consideration. They do want to make it into the enterprise space. And the enterprise space does require some level of security maturity.

So they have to start thinking about how they do things in a secure way by default and they create those best practices . In terms of how you create the patterns, it's just, unfortunately, it's a lot of man hours. It is what it is, you know, different languages have different structures. They need to be created in a different way.

We try to optimize as much as everyone else but a lot of the time, the way you have to deal with it is to put the effort in , take some time off to look at what's popular and decide which one you want to focus on. It's not a case of just expanding again, expanding sideways. We constantly have to expand upwards, kind of like supporting later and later versions of a language and that it just comes with whatever. On our side, I might have a great team that can do a lot of the work. They're getting better at it. You'll, you'll be surprised that the more people do the security analysis, the better they get at it, just like they would with Threat Modeling. And they're able to say, hey, you know, this language is not really doing a lot with anything else other than manipulating databases, so I'm just going to start looking around for research being done by people. I'm going to look for you know, talk on the topic. I'm going to look on papers on the topic of this language and writing to database if there's anything there and that's how you handle it. That's how you build a lot of the knowledge, a lot of the information you need in your Static Analysis tools to get where you want.

And, a lot of the times that we're seeing lately, I think people, they want a lot more focused things in terms of remediation. So a part of the time we spend is not necessarily figuring out what we need to look for because the community, a lot of times, exposes those and vendors do. But while we spend a lot of time writing the correct article remediation, information, the advisor information to help a developer understand it.

Because whether people want to accept it or not, this stuff needs to be simple and needs to be usable by a developer. Sometimes we get it right. Sometimes we get it wrong and we have to go back and redo it. But that's where we spend a lot of our time, is writing the articles that make it easy for them to work with. The rules themselves, you write a few and then you get used to it. 

Abhay - Right, yeah. I mean, absolutely. I think especially with the state of the languages and suddenly, I mean, especially , yeah, just over the last two years, I myself have seen the sudden expanding landscape in terms of languages. The other thing that also comes to my mind quite a bit with this is that today , I think, Static Analysis used to be, at least when I say used to be in, in some cases for some tools it still is but it used to be extensively language focused, right. You are looking at JavaScript language vulnerability, Java language vulnerabilites, but what today people, I think want are framework level vulnerabilities. Hey, how do I find vulnerabilities in Vue.js or Angular or, you know, Django or Flask or...yeah node at least a little bit more platformy than framework. But what I'm trying to say is that they want... So for instance, let's say I'm writing cloud code with Python and Amazon. I want to find library issues with Boto3. How am I using Amazon's Boto3 and introducing vulnerabilities in my Amazon stack that I've written in Python.

That's what I think people are looking for. And that's why you see a lot of these , you know, these query tools have tried to come into that space as well. How, how do you address those? I mean where do you , I mean, what kind of strategies do you adopt for such things, especially because, I think, that's,  it's not a need first of all or do you see that as a need? And how do you address that? 

Florin - Yeah, absolutely. You do have that as a need and [pause] every framework will do things in a slightly different way. As I said, in my previous answer, luckily a lot of these frameworks are coming up, they kind of come secure by default. So doing bad things, you really have to go out of your way and mess with some things to do so but there are still a lot of cases where you can do that.

Frameworks come in not much different than any other language because they tend to be so different in the way you use them. And, a lot of times it's a set of brand new APIs. We kind of have to do the same work and have to do the same research. From our side, we kind of bundle all the frameworks under the same language.

We can't really say upfront which one you're using. It's not always visible. Sometimes you might not have the library's attached or isn't fingerprinting them, is probably not the easiest thing to do. So we run the rules, if we get a hit we'll... we'll know, hey, it's this framework and we'll present it back to you, but it is very important,

I think it's very important to create the rules specific for those frameworks. Sometimes, you'll find that a generic rule can find something in a framework but where you would struggle is what I mentioned, in the previous answer, the remediation advice and the information about what the problem is because this is the framework specific.

It'll have a different way to be faced in a generic problem. You know , better thinking at SQL frameworks that would have vulnerability in Java, but an execute query would probably have a different fix than a broken Java SQL integration library that execute query. And, when you have a framework, we want to investigate the framework. We want to understand what is the correct way of doing it in that framework, because if you do it in the framework's way is probably the safer way to do it than writing your own code and building your own thing. One of the interesting talks I always have with people in our research lab and specifically the DAST team that I talk to once in a while is they keep seeing people trying to implement their own Cross-Site Scripting Protections , their own Cross-Site Scripting Protection libraries and code. And the amount of times it works, it's, yeah, [laguhter] it's hilariously low and they get it wrong so, so many times, even with the OWASP Cross-Site Scripting cheat sheet and all of that, it's difficult.

So what we try to say is like, hey, you're using this framework. Here's how you do it. Just don't use this dangerous flag in your construct. Don't say execute script equals true for every single thing you print on screen, just because you needed to execute some scripts somewhere in your code. So just disable that and you'll be safe.

It's the framework. So one way of doing things safely, but then it's important to look at the frameworks and we try to do it ourselves.  [pause]  I think JavaScript is the number one offender in here. You know, I expect someday I'll have to put up with a JavaScript framework just to cook my dinner because my new stove will use JavaScript.

It's coming in so many different flavors , developers like it. I think that frameworks are important. 

Abhay - Yeah, no, absolutely JavaScript. Yeah. I think that, I think there's probably like one JavaScript framework released every day or something. I think it's, it's, it's so ubiquitous. It's all over the place. Right. So, yeah. Which, I mean, just a quick trivia question, which is your worst or which do you feel according to, I mean, based on your Static Analysis experience, which is the toughest vulnerability to identify or class of vulnerabilities to identify? 

Florin - Ah, that's a, that's a tricky one because it depends on the application you're dealing with. I would say something really difficult to identify for Static Analysis in general, it's backdoors and it's frameworks security tool. 

Abhay - okay. 

Florin - It's not necessarily a vulnerability. I wouldn't call it. It's not a type. Right. So I don't know if it's fair to answer the question that way because the way I like to put it, it's backdoors.

It's perfectly reading code. It's perfectly secure code that does really bad things. They will not let credentials and validate it to go through. So backdoors are nearly impossible to find. But if I were to choose one of my favorites that is somewhat tricky to find by some tools and SAST is really good at, I would say probably like a blind SQL injection . For DAST, it's quite difficult to get a blind SQL. For Pentesters, it's all about the timing and trying it and retrying it and do this progressive staged query, while Static Analysis can just look at it and be like, hey, it's simple writing to SQL. You're not sanitizing data. That's a blind SQL injection. Or I think some people would call the second or SQL injection depending on where you're looking at, which place you're looking at it from. I think that's an interesting one for Static Analysis.

Is it the most difficult one? I don't know, I have my engineers to tell me which one is that. I'll, I'll have to go back to them and ask which one they hate the most, but it has to be something in JavaScript. Definitely something in Javascript. 

Abhay - It's probably Cross-site Scripting is what I had imagined because I can't imagine actually having a very.... I can't imagine being too confident writing a Cross-Site Scripting, identifying Static Analysis rule myself. I mean, unless it's like dangerously set in our HTML or something, I mean something obvious like that. 

Florin - Yeah. I think it depends on a framework. In some frameworks will have a good time of saying you just disable the Cross-Site Scripting protection. It stays right there on the tip, Cross-Site Scripting protection you set it to false. Yeah, but in some of them, just like you said, it will be difficult to find it out because you're getting the data from somewhere, you're passing it through a different application layer and then you bring it back out, putting it back to the user. So it's, it's difficult, but a lot of them.

Abhay - that's probably worse with DOM Based I think. With DOM Base, it's probably even worse. 

Florin - Yeah, I would struggle to tell which one is the worst but there is a chance Cross-Site Scripting is out there. I mean on the other hand, Cross-Site Scripting is amazing for Dynamic Analysis. Dynamic Analysis can just rip your applications to the treads with a million different encodings for, for a payload and find stuff. For Static Analysis,

it would be like, I don't know if this render is right or not. I think it does print stuff out to the user as like maybe. 

Abhay - I think there are too many interaction touchpoints, I think with Cross-Site Scripting, which is where I would imagine that that would probably be the worst because I can't, I can't imagine being very confident writing a Cross-Site Scripting rule.

But anyway, 

Florin - I love the frame. We started doing a lot better. I'll just say this, you know ,  shout out to whoever has been building protections into their frameworks for Cross-Site Scripting and some of the big, really big vendors have been doing this. The browser is themselves, you know, they're, they're a lot better at saying like, hey, hang on a second you're trying to execute another JavaScript. Like, hang on a second. Give me a second. So I think people are getting better at that. The one I would like to absolutely see gone if it was after my preferences would be, I would just kill out the SQL injections. You know, Bobby Tables has been around for a long time.

Abhay - 1998...    [laughter]

Florin - if you don't know who little Bobby Tables is, you might, you might want to look, look at that, but you will want to do that. 

Abhay - Yeah. So, I mean the other thing that I always wonder, especially and  I thought of asking this question, especially once we scheduled this was obviously you've seen the language market evolve.

We've seen Static Analysis Tools evolve in general. How they evolve in different types of tools. How have you seen the customer landscape evolving? How have you seen your customers changing in big and small ways? What are those things that you've noticed in your , you know, when you talk to them, obviously you have some touch point with them and you talk to them about different types of things, how you've seen them involve or how are they grappled with this problem?

Florin - I think the immediate thing we've seen after the Agile and DevOps approach came to market. And all of that was , hope I'm using this term, right they said ‘democratizing development’. And they said, whatever framework you want to use, use it. Just let us know about it. We'll look at it before I have some analysis and then say if you're good to go or not, and what we've seen a lot is they went from, hey, were a Java shop or a .NET shop or were a PHP shop too, aah!! we'll do a bit of everything.

You know, we need the tool to do a bit of everything and we're like, okay, so what does everything mean? And then you go through the lesson. It's like, hey, we have Java, we have  .NET. We have PHP. We have JavaScript with node, with view and react. We have Cobol, you know, we still have a mainframe.

We do some Cobol. We have Python because we do some artificial intelligence projects. So  I've seen a lot of languages coming into the environment. A lot more than we used to do before. A lot more than [pause] we thought will happen. In, in reality without company, you still want to standardize and development platform, but now a days its like, no, no, you know, they have their own choice.

They can pick whatever they want and they'll use whatever they want and you kind of have to help us secure it. And it's fair enough, you know, for us, we're happy to do it. We had some technologies in place that could get them started with straight away. And then later on bring Static Analysis to cover all of those languages.

What we did see is it's kind of more important for them to have a set of tools to leverage. And when I say set of tools, I'm not necessarily referring to different vendors but, you know, some of them will be Static Analysis. Well, you just brought a new language for some web, no one supports it, whether you're going to do. So

that's where we recommend, you know, have a set of tools at your disposition, Static Analysis, Dynamic analysis , share some of that knowledge with your developers, let them know and give them a tool that works for them and find the model that makes it work. And everything we found is they started focusing a lot on specific high-level threats , not specific but they're really wanting to get, hey, just I understand that can not be fully secure. And they kind of went to this approach from Waterfall because Waterfall kind of gave you a nice, comfortable piece at the end where you could spend time and say, you know what? I will go ahead and I will do my security testing at this point. I'll have a Pentest, I'll fix stuff and then I'll release but because you're now moving to hey production, you know, feature production, feature production , accelerating the cycles and accelerating the release.

We are seeing them a lot more interested in the, hey, let me catch really important things. They're really dangerous ones. I wanted to catch them. I want to make sure I don't make any mistakes there and I'll have some appetite for a security risk. It doesn't always happen for that appetite to be big. I think most will have a, quite a low appetite for risk but they do understand, hey, I cannot be a hundred percent secure but I want to make sure that the things that would really hurt me and my customers are being taken care of.

I'm handling those in my development and they never make it to production. So we went from this place where you had security teams kind of being responsible for a security because they're running it all the way at the end of the day, the code reviews to more, hey, you know, developers kind of going to do this throughout and we're going to focus on finding the really high priority ones and fix those as we do so they don't make it to production.

The one thing I do like seeing and I kind of like seeing this to be honest because it includes developers to this process and hopefully they're not gonna release those really, really dangerous things into the wild while allowing their business to stay competitive. They have to, I'm sure some people will take shortcuts.

They'll become very famous and then your data gets exposed. My data gets exposed and we're now.. we're now better for it. And this was good to see security developers, but one of the trends I didn't like seeing is when companies moving to DevOps it put a weird model in place where security is kind of like a little bolt-on.

Someone does some security but [ pause] they don't own the code. And they're not really part of the development team. They're not security either. And it's like a tool you're putting the code through. And developers get very frustrated. And when they get to ask to fix everything because they get this long list of vulnerabilities, they're asked to fix, and they're like, first off, I don't know what this is, second off they're all false. Prove me wrong. Otherwise, they're all false. And it goes back to what I was saying. You know, I didn't like that people made the tools the process . I like it when people make the tools fit your new process, which developers should be scanning. I like when they make the tools part of the process and define what they do, but some places I've seen, they kind of made the tools the process and developers were not happy. I'm happy to report that most of those that I've seen are changing but it took a while to listen to developers and understand how that needs to be done. But I'd say the biggest change, languages, different languages. Bunch of different frameworks , then building languages and frameworks themselves. I've seen a few of those cases. So switching from, hey, you need to be really good at Java to support me to, hey, you need to do all of these languages to support me, which has been, has been a shift that I think a lot of the SAST vendors have to go through. 

Abhay - What, what impact do you think at least from your company's perspective has the cloud had on, on you folks? As in, has it , I mean, some complexity, of course, may have been brought in, but has it given you some opportunity? What has changed for you both from a weakness and opportunity perspective?

Florin - I mean, for us, a big opportunity of the cloud is we can offer things as a service all of a sudden, and when you offer things as a service, it's a much easier way to improve on things. It's a much easier way to kind of fail, fail fast. It's a concept that everyone follows, you know , it doesn't mean necessarily you fail, but you push something out and you get feedback on it, then you can improve it as you go along. And that was kind of the nature of the cloud for us. It allows us to do that while, you know, with the on-premise you have to be a bit more thorough when you push out some, some things. 

So that was a great opportunity for us. Some of the challenges that came was, all of a sudden, we're starting to look at Infrastructure as Code which, you know, it's, it's fair to say. We kind of saw coming but it probably came to market a lot faster than we expected. And it then became popular a lot faster than we expected because some of the pretty popular companies benefited the cloud native things. You know, they've benefited from this way of doing things in the cloud. Like, I'm just going to run a few lines of code and it's going to automatically deploy stuff for me, it's going to deploy a server, and it's going to deploy a new machine for my project.

And at some that we need to learn to handle. It's one of our main projects right now, understanding how we can do this with Static Analysis because surprisingly, if it's code, if it's a configuration file that we can scan it, we can find stuff in it. So why not do it? You know, it's an opportunity for us to help developers even more.

Abhay - And the... thing that you only have to go with the YAML, right, you just start to scan YAML?

Florin - sorry, what was that? 

Abhay - In most cases [Laughter] ....You just have to scan YAML in most cases.

Florin - Oh, yeah... you scan the YAML.  You scan the, I guess, you know, Kubernetes use YAML and Docker files are YAML. So I'm still learning how the YAML is different, Docker configurations and how I can do it. But yes, it does bring some challenges, right,. As I said, at the beginning, with Application Security, you find out, you have to learn about something else.

You have to learn something new and it's good. It does sometimes make us take a step back and reconsider. Okay, what are we doing? You know, do we want to boil the ocean looking for this one vulnerability? Whether we want to kind of do like a high-level surface thing and find there's been things there's been things as you can.

And it's a lot of questions that makes us ask again, you know, you take another step back , but the best of opportunities for us, you know, our Machine Learning would have been really, really hard to do without the cloud. Because in the cloud we can, we can look at how it works. We can talk to customers, we can adjust ourselves.

One of the things we have to do with Machine Learning is we have to supervise it. We don't want our algorithm to think that Cross-Site scripting is not dangerous because everyone is marking those findings as noise. It's the last thing you want to do with it. So we had to supervise it, but the cloud allowed us to supervise training, see how it behaved , look at the impact and then make it available for people. On the other hand, you know, the cloud gathers a lot more applications being built a lot faster. A lot more technology is becoming popular in the Infrastructure as Code. And, we know how to look at it and do our analysis and research it and start driving rules to find those things as well. Luckily, YAML is very easy to parse unlike JavaScript. And  [pause] again, it does a lot of really good things by default. I was looking at the OWASP article, kind of like the top issues you can do. And if you look through this, most of the times, people just want to do something more and then they enable right to this for something.

And things as basic as that, like a look. By default, I'm running with read-only; don't, don't make me write to disk. I don't have to. And I'm showing my playing cards here, which becomes a bit of a habit on COVID nowadays, but it allows us to do that. The cloud allows us to look at new things and learn new things for ourselves.

And you know, ourselves can benefit from it as well. We can make our scanning faster. We can make our machine scale up more, you know, giving us faster results back. It allows us to look into Machine Learning, more powerful systems for Machine Learning and all the different swarming things and all of that.

There's some cool research going on with, with all of that and what can be done. So the benefit, whether everyone else gets, the benefits of the cloud . The Challenge is, you know, having to deal with this new world, Infrastructure as Code. 

Abhay - Right, right. Right. So last question Florin before we go. [pause]  If you were to highlight three things, especially given this current circumstances that companies are doing wrong that if they did right, they would, they would get, you know, let us say massive benefit from Static Analysis. What are those three things? 

Florin - One of it, we've probably mentioned a few times, but learn to do some Threat Modeling, where you teach your developers to use some basic Threat Modeling. If they understand the basic concepts of a few issues, they will be in a really good position to look at something and say, you know, all of this, I don't think it's a priority for us.

So we're going to push it to the side. It's going to help us deal with the noise. So a bit of Threat Modeling education for developers or get some people to help them with Threat Modeling. I think number one, we'll give you some good value out of it. Convert those into policies and that's pretty much your application security playbook

as we said . The second item I had in my mind and it, it kind of elevated me was and..now now I'm drawing a blank because I hadn't... so nice in the lineup to move into from Threat Modeling. Ah.. security champions , start making security part of the development and start talking to them and start asking a few questions.

Just start with, what about security? Just ask one of those questions to see if anyone has an answer to it. If your development team has an answer to what about security, it means, that's good. They could, they could ask you which security, which is great. That means, okay, they've considered some things.

They can ask you what security, and that's a big concern because there is no security but you know that right, you know, there's nothing in place but start looking at getting some security champions as part of your development teams. There were some really good DevSecOps groups out there. They will be talking about how you can make security part of it. Then make security a bolt-on to your practices. It's difficult. It's difficult for developers, more than anyone else. You think the security person that will have to run all the scanning is the one that struggles most, where reality is a lot of times you're pretty good with scripts.

We know how to automate things. We'll be able to put your code through the tools, get the results out and that's fine. But what happens then if, if you don't have that Threat Modeling, you won't know what to look for. So you look for everything, which will get developers angry. But if you cannot do the Threat Modeling and you still have to send the full set of results back to developers, at least have that security champion to do it with the team. They'll be able to go through that and say, look, you know, we think this is, these are the threats we want to target in your application. Realistically, it's what we can humanly do as part of our development cycle. So do that as well. And if I were to pick a third one , just be prepared to reconsider what you're doing. Honestly, I've seen a lot of people being in the mindset of we want to do this.

We need to do this. It needs to be done this way before doing any analysis , you know. I'm a strong headed person. I'm sure there's other people like me out there and it's like that's how we do it, it's the best idea. Just be able to reconsider things. We've seen projects where they want to do things in a specific way, turns out that that never got off the ground. And we had to go back and reconsider, go back to the drawing board. I took a while to convince them to follow the model but after they followed the model, we became really close friends because we found out that something they can do, they found out that they can make the tool of part of the process and not make the tool the process. And going back to that thing, we mentioned earlier on, and then they made it work for them. And in the end, it turned out that they were happy. They can use the tool, they can get the things they want out of Static Application Security Testing, and their developers were happier that, hey, it's a workload we can manage.

We understand why we have to do it, which was the most important thing. Understanding. And they found out that, hey, we can now build some security champions and some security groups and developers. You'd be surprised how quickly they can become self-sufficient when these things, you know, they learned to deploy agents through code.

They learned to do a lot of crazy things over the last few years. And security is not that much of a challenge if you just give them the time and the right pace at which they can learn things [pause]. And those would be the top three. 

Abhay - Thank you. Yeah, I think the quote from this interview would be “make the tool part of the process and not the process.” So I think that was the, the biggest take home, I think, for a lot of folks. Don't, don't think of the tool as the, be all and end all of things.

And I think that's very important because a lot of times, in DevSecOps people think in terms of tools and automation all the time, and they don't really think of people working together. And I think that's very importantly...  

Florin - Absolutely.. Yeah. I remember talking to someone about .. we always want to do more of the tools. Right. You know, self-driving cars. Awesome. All of this, the thing is even with a self-driving car, as great as it sounds, you still want to know why it's doing certain things. I want to know why my car suddenly is going right,  [pause]  and in the middle of the road and the basic concept is yes, it goes right because you switched lane because you want to take a right later on.

But if you don't understand all the time, you're like, why is it doing this? Same thing with security. Don't just expect the tools to do things for you. Explain why you want to do some of these things. And myself, you know, I sell tools. It is the honest truth working for a company that focuses on this but we do want our customers to be successful because at the end of the day, my credit card details are with them. My personal details, my entire life is stored in a digital format with them. So when they fail something, I feel it, you know, it has repercussions on me; I'm the one getting a TV bought somewhere in a random part of the world or my credit card is the one that's used to purchase cinema tickets for a group of people somewhere in the world.

And it's important to to start thinking that, hey, security is something for all of us. We don't realize it, but we do security every single day throughout our lives. You know, security of our own person, just don't put your hand in boiling water and code is going to be a bigger part of our daily lives with IoT coming with a lot more gadgets around us. It's important that we start thinking like, okay, yeah, what can I do to prevent this from posing a threat, a psychological thriller on that or whatever threat there is. Thing is..the most interesting story with security posing, a cybersecurity posing a physical threat came from, I'm not sure if you remember the Ashley Madison hack.

Abhay - Yeah, 

Florin - it was the, the dating, whatever agency it was. And I think there's like one or two people that committed suicide because of it. 

Abhay - Yeah. 

Florin - Which was quite a sad thing to see that, hey, all of a sudden, this intangible thing, which is the software, which is the code can cause repercussions in the real world.

And we need to make sure that we don't do that. We are as good as we can with security, but you know, let's educate ourselves. Let's take it slow. We were never taught security in school. I don't think anyone did. We learned on ourselves, on our own. And it's, it's a process. Let's make sure we make it as enjoyable as possible.

Abhay - Right. Absolutely. Thanks so much Florin. And thanks for being here with us and thanks for sharing your thoughts. I think it was very interesting to hear different perspectives. I hope you had fun too. Thanks so much for being....

Florin - Absolutely. As I said, I liked talking and I like chatting with people. I'm looking forward to be here back in the next session, in a year's time to see how everyone else is doing.

Hopefully, everyone's traveling or maybe we get to meet at one of these conferences. I think we're supposed to cross paths a few times if you were going to all the main conferences, but then you know, like I've, I've heard someone saying recently until next time, you know, everyone stay positive and test negative.

Abhay - [laugher]  Yeah. That's a good one. That's a good one to live with. Thanks so much Florin. So that was Florin from HCL AppScan. I really liked chatting with him. It was a lot of fun and until the next episode of AppSecEngineer, thanks everyone. Thanks for watching. Bye.