NDC was really an awesome conference and I got to meet some really interesting people there. I was satisfied with most talks I attended and got some new influences.
As usual this conference struggles with inequality between the amounts of male to female speakers. Some will argue that this is just how things are since the conference call for papers are open to everybody. I don't really think this is an excuse since we our self are responsible for building the society we want to live in and that might mean initially putting some effort into creating equality. There was a code of conduct for the conference which is a step in the right direction.
The venue at NDC was really nice, the food was great and, some really excellent coffee was available. All in all I’m really happy I got a chance to attend. All the talks will be available on Vimeo.
Sunday, June 21, 2015
Friday, June 19, 2015
Mob Programming – A Whole Team Approach by Woody Zuill
Another concept that I’m completely new to, NDC is fantastic source to for getting interested in new things.
Mr Zuill starts the talk describing what Mob programming is: All the brilliant minds working on the same thing, at the same time, in the same space on the same computer.
During a day there is one driver who sits at the computer and the rest of the team is navigator. The driver inputs what the navigators explains. Anyone can take a break when they want to. If the driver wants to express an idea then that person needs to switch. The driver gets rotate on fixed cycle.
Benefits are instant review of both code and design, good learning opportunities, and, team resiliency. To work this way focus should be paid to individual and interactions. The team should exercise kindness, consideration, and, respect and make each other aware when they fail in these areas.
Productivity remains high due to low latency in communication and almost no technical debt.
Mob programming certainly seems like an interesting technique, excellent talk. There is a website for mob programming.
Mr Zuill starts the talk describing what Mob programming is: All the brilliant minds working on the same thing, at the same time, in the same space on the same computer.
During a day there is one driver who sits at the computer and the rest of the team is navigator. The driver inputs what the navigators explains. Anyone can take a break when they want to. If the driver wants to express an idea then that person needs to switch. The driver gets rotate on fixed cycle.
Benefits are instant review of both code and design, good learning opportunities, and, team resiliency. To work this way focus should be paid to individual and interactions. The team should exercise kindness, consideration, and, respect and make each other aware when they fail in these areas.
Productivity remains high due to low latency in communication and almost no technical debt.
Mob programming certainly seems like an interesting technique, excellent talk. There is a website for mob programming.
How Do You Scale a Logging Infrastructure to Accept a Billion Messages a Day by Paul Stack
This seminar is a description on how Paul Stack changed a legacy solution using SQLfor storing logs to a more scalable solution.
To replace the SQL solution they went to the ELK Stack which is made up of ElasticSearch, LogStash, and, Kibana. In the beginning a developer took a single JAR file and ran it, this worked well but had some stability issues.
Next iteration Redis was used as transport medium for the logs before it was pushed to ElasticSearch. Due to problems with Redis overflowing they moved to Apache Kafka. Kafka is a high-throughput distributed messaging system and is backed by ZooKeeper which is basically a key-value store.
The following iteration they tried to improve the server structure however this mostly increased the complexity of the system without improving the system. The system could new handle the peak load of 12 billion messages per day.
Mr. Stack recommend reading Jepsens database series for choosing technologies. A lesson learned from the project is to not re-invent the wheel, the simple solution would have been to buy Splunk however during the development they learned a lot about their system.
I’m not a such deep into DevOpts but I found this talk very informative.
To replace the SQL solution they went to the ELK Stack which is made up of ElasticSearch, LogStash, and, Kibana. In the beginning a developer took a single JAR file and ran it, this worked well but had some stability issues.
Next iteration Redis was used as transport medium for the logs before it was pushed to ElasticSearch. Due to problems with Redis overflowing they moved to Apache Kafka. Kafka is a high-throughput distributed messaging system and is backed by ZooKeeper which is basically a key-value store.
The following iteration they tried to improve the server structure however this mostly increased the complexity of the system without improving the system. The system could new handle the peak load of 12 billion messages per day.
Mr. Stack recommend reading Jepsens database series for choosing technologies. A lesson learned from the project is to not re-invent the wheel, the simple solution would have been to buy Splunk however during the development they learned a lot about their system.
I’m not a such deep into DevOpts but I found this talk very informative.
Thursday, June 18, 2015
Impressions day 2 of NDC
James Mickens closed out day two with his awesome standup routine Not Even Close: The State of Computer Security. The rest of the evening is a party with some live bands.
Today has been really good and even though not all the seminars I attended will be applicable I feel that I got a lot of inspiration.
The food is pretty good considering that it is catered and there are several selections to choose from. The only complaint I have is that there is no dedicated slot for dinner so you either choke down your food in less than 20 minutes or miss a seminar slot.
All in all I think I can say that NDC is definitely worth a visit.
Today has been really good and even though not all the seminars I attended will be applicable I feel that I got a lot of inspiration.
The food is pretty good considering that it is catered and there are several selections to choose from. The only complaint I have is that there is no dedicated slot for dinner so you either choke down your food in less than 20 minutes or miss a seminar slot.
All in all I think I can say that NDC is definitely worth a visit.
Ten Simple Rules for Creating Your Own Compiler on the CLR by Philip Laureano
Don’t we all secretly dream of constructing our own language?
The good news is that everything does not need to be written from scratch. The extremely quick run trough of a compiler is that the code is parsed into a parse tree which is converted into an abstract syntax tree which is compiled to an assembly.
Don’t write your own parser, use a prewritten one for instance ANTLR. When constructing the grammar rules it is important to avoid circular loops since that can cause an infinite parsing recursion. The parser that is generated is quite unreadable.
Going from the parse tree to the abstract syntax tree requires high usage of the visitor pattern. The abstract syntax tree then forms the heart of the compiler. Mr. Laureano has used LINQ expression trees to handle the abstract syntax tree.
Mr Laureano’s toy language, Not Quite Lisp™, is available on GitHub.
I wish I could say that this talk taught me all I needed to construct my own language but alas it turns out I’m either to dumb or the problem is more complex than an hour’s seminar can explain. It was fun to listen too though.
The good news is that everything does not need to be written from scratch. The extremely quick run trough of a compiler is that the code is parsed into a parse tree which is converted into an abstract syntax tree which is compiled to an assembly.
Don’t write your own parser, use a prewritten one for instance ANTLR. When constructing the grammar rules it is important to avoid circular loops since that can cause an infinite parsing recursion. The parser that is generated is quite unreadable.
Going from the parse tree to the abstract syntax tree requires high usage of the visitor pattern. The abstract syntax tree then forms the heart of the compiler. Mr. Laureano has used LINQ expression trees to handle the abstract syntax tree.
Mr Laureano’s toy language, Not Quite Lisp™, is available on GitHub.
I wish I could say that this talk taught me all I needed to construct my own language but alas it turns out I’m either to dumb or the problem is more complex than an hour’s seminar can explain. It was fun to listen too though.
Making .NET Applications Faster by Sasha Goldshtein
Any person who does not care about performance of their components is a horrible person. Shun that person like the plague. The seminar focused on three areas for improving performance:
Another collection group is trees; examples are SortedDicitionary and SortedSet which have efficient lookup but they have an even bigger space requirement per item than linke lists. There are also associative collections such as Dicitionary and HashSet which is somewhere in between LinkedList and Arrays in space requirements.
There are use cases when the existing collections does not fit well, for instance finding words that begin with a specific prefix. Structures that handles this kind of scenario are called Tries.
The key point is that for certain scenarios using a custom collection can boost performance.
Startup can be separated in cold startup and warm startup. Cold startups are when the application has not executed since last reboot, this startup time is dominated by disk I/O operations e.g. loading assemblies and similar. For warm starups JIT compiling and similar is the biggest problem
To improve startup time NGen can be used to compile the .NET code to native code which will speed up load times. Another trick is to enabled Multi-Core background JIT by using ProfileOptimization.SetProfileRoot and ProfileOptimizations.StartProfile. Another solution is to use the faster RyuJIT compiler, however it is not released yet so it might not be 100% stable. It will be part of Visual Studio 2015 and .NET Framework 4.6 release.
To improve cold startup all the assemblies can be merged together and this can be done with ILMerge. Another approach is to use .NET Native that compiles everything down to a single native executable. This removes the need to having the .NET Framework installed on the target machine. However .NET Native is only available for Windows Store Apps.
For optimizing GC handling there are many performance metrics:
Finally: don’t use finalization.
This has been the absolute best for me so far at the conference
- Working with collections
- Improving startup times
- Reducing GC pressure
Another collection group is trees; examples are SortedDicitionary and SortedSet which have efficient lookup but they have an even bigger space requirement per item than linke lists. There are also associative collections such as Dicitionary and HashSet which is somewhere in between LinkedList and Arrays in space requirements.
There are use cases when the existing collections does not fit well, for instance finding words that begin with a specific prefix. Structures that handles this kind of scenario are called Tries.
The key point is that for certain scenarios using a custom collection can boost performance.
Startup can be separated in cold startup and warm startup. Cold startups are when the application has not executed since last reboot, this startup time is dominated by disk I/O operations e.g. loading assemblies and similar. For warm starups JIT compiling and similar is the biggest problem
To improve startup time NGen can be used to compile the .NET code to native code which will speed up load times. Another trick is to enabled Multi-Core background JIT by using ProfileOptimization.SetProfileRoot and ProfileOptimizations.StartProfile. Another solution is to use the faster RyuJIT compiler, however it is not released yet so it might not be 100% stable. It will be part of Visual Studio 2015 and .NET Framework 4.6 release.
To improve cold startup all the assemblies can be merged together and this can be done with ILMerge. Another approach is to use .NET Native that compiles everything down to a single native executable. This removes the need to having the .NET Framework installed on the target machine. However .NET Native is only available for Windows Store Apps.
For optimizing GC handling there are many performance metrics:
- Performance Counters
- Event Tracing for Windows
- Memory Dumps
Finally: don’t use finalization.
This has been the absolute best for me so far at the conference
Short Between Seminars Post
Something that keeps beeing mentioned on the conference but I have not brought up in my other posts is Conway's law. There is also several references to the book The Inmates are Running the Asylum.
Subscribe to:
Posts (Atom)