top of page
Search
conkufejac1979

CoPilot on test: A comparison of Copilot and other code completion tools



By using GitHub Copilot and sharing your feedback in the feedback forum, you help to improve GitHub Copilot. Please also report incidents (e.g., offensive output, code vulnerabilities, apparent personal information in code generation) directly to copilot-safety@github.com so that we can improve our safeguards. GitHub takes safety and security very seriously and we are committed to continually improving.




CoPilot on test




Just like when you write any code that uses material you did not independently originate, you should take precautions to understand how it works and ensure its suitability. These include rigorous testing, IP scanning, and checking for security vulnerabilities. You should make sure your IDE or editor does not automatically compile or run generated code before you review it.


You should take the same precautions as you would with any code you write that uses material you did not independently originate, and should take precautions to ensure its suitability. These include rigorous testing, IP scanning, and checking for security vulnerabilities. You should make sure your IDE or editor does not automatically compile or run generated code before you review it.


Bringing in more intelligent systems has the potential to bring enormous change to the developer experience. We do not expect GitHub Copilot to replace developers. Rather, we expect GitHub Copilot to partner with developers, augment their capabilities, and enable them to be more productive, reduce manual tasks, and help them focus on interesting work. We also believe that GitHub Copilot has the potential to lower barriers to entry, enabling more people to explore software development, and join the next generation of developers. We are working to test these hypotheses with both internal and external research.


I pasted those comments into my test file inside the existing list of parameterized tests, then wrote each test by adding a newline beneath a comment and hitting tab until Copilot had written the test for me.


Create unit tests An important use case of GitHub Copilot is to take some of the drudgery out of writing unit tests. Below, we already have an implementation of a function that computes the common prefix of two lists, and we want to test that. To do so, we import the unit test package, and then we start writing a test function to let Copilot generate the asserts which we accept just by hitting the Tab key.


With these two files you have all you need to deploy the string reverser application with Copilot. As covered in the introduction to Copilot, you can use the copilot init command to run a wizard that detects the application and automatically deploys it. At the end of the process, Copilot gives you a URL for the application.


Now whenever a developer pushes a code change, the code is deployed to the test environment first, the integration tests are run against that test environment, and only if the tests passed the code is deployed to the production environment that customers are using. This shows up as some new steps in the pipeline status:


This is a good thing! It means that the test is catching the issue. After the issue is fixed you can have confidence that the tests will prevent this bug from ever reoccurring and reaching production again.


Writing end to end tests is difficult and time consuming. It takes forever to dig into the page code to build the right XPath or locator. And even when these are found, they are likely to change over time as the product continues development.


Pair programming is an extreme programming technique where two developers work together on one computer, one acting as the pilot and the other the copilot. The pilot is on the keyboard while the copilot helps navigate, checking the logic and patterns, while weighing against alternative solutions. While two developers producing one piece of code sounds expensive, the quality of the final work can help avoid added costs of code review and QA.


When you type this function declaration into your js file, you will see that GitHub Copilot starts to fill in what it believes to be the required implementation. The code that GitHub Copilot provides converts the begin and end inputs into dates, determines the days between the two dates, then returns the result. This initial test demonstrates how GitHub Copilot is able to help suggest and generate code.


To have a consistent testing environment, I set up two directories: one with plain JavaScript (no dependencies) and one with a Next.js application, so I could play around with React and an established React framework briefly.


Merlin plans to deploy its co-pilot on cargo routes in New Zealand first on Cessna Caravans, where relatively empty skies and rough terrain make it ideal for testing aerial supply lines. This will be a civilian air route, but the company intends to take lessons from these routes to flying US Air Force C-130 planes on cargo runs.


I joined Containers from the Couch for a live show with Adam Keller and Paavan Mistry to talk about using AWS Copilot to setup a deploy pipeline thatbuilds and releases a Docker container to AWS Fargate on git push. I also added integration tests to verify thatthe application is behaving as expected. This live recording and demo is a counterpart to my AWScontainer blog on the same topic.


Adam: Yeah I think one thing with Copilot when you run your first time usingcopilot init it basically initializes a skeleton structure for yourapplication and when you think in Copilot you have an application whichencompasses multiple services right? So your application will deploy a loadbalanced service in this case which has a Docker image that needs an ECRrepository. This all happens at that initialization phase right Nathan?


Paavan: So I have a question here Nathan. So when you click or when you type incopilot init where does the manifest have to be in the same directory forfor it? Where does it need to be?


And this goes through a lot of different ways that you can test all the way toreally rigorous test environments. The test environment that I want to set up,in this case, is an integration test which is this first integration test discussed inthis section right here.


So all these ones that are standard test, standard prod, these are all Copilotmanaged Cloudformation templates and the one that I just picked right here is thepipeline so this is actually creating a CodePipeline for my service.


Like I remember back when I used to run Jenkins myself, and as Adam can testifyrunning Jenkins was a pain. I actually had a build box that wasin the office sitting over there on a desk running Jenkins and that was theresources that I had to manage. Every once in a while I had to go inthere and apply patches or like figure out why Jenkins had crashed, right?


So let me show that the this phase first which was the actual CloudFormation deployment.I can see if I click through here the events. I can see as each of thesedifferent resources got updated, the task definition and the service got updated to a newversion of my service. And then I can actually see the testoutput by clicking on details. And as I scroll down through here I cansee that it installed my test dependencies. And thenit started running my actual tests against the service whichin this case was testing to see if I wasable to reverse the string.


But I want to say one last thing on on that question fromfrom Mike in chat. I think Copilot is going to get you theenvironments and everything you need to get your application fromtest to production and once you have that all deployed use git.


I wanted to keep things simple for my initial tests, so I created a new, empty project in PHPStorm and created a class called Utilities (yeah I know) to define a few static helper methods for some contrived common tasks I might want to perform on strings and arrays.


I need to start typing something in the function body this time - there is no automatic suggestion of a complete test when I hit the enter key to start a new line. I begin with a basic $input = 'StringToConvertToCamelCase';


Enough with the small stuff. I want to know how much Copilot can help me with real-world code. I also want to test its limits; can it effectively write an entire class for me - one which is really, genuinely useful? 2ff7e9595c


1 view0 comments

Recent Posts

See All

Comments


bottom of page