C# vs Python – Part 2

I have with C# for easily about 8 – 10 years and I love the language. I have worked with Python for much less time. Probably about 1 year in total and I also like the language. There are a lot of discussions on the web in regards to differences, pros and cons betweens specific languages. In this series, we’ll regroup comment, ideas and informations on that topic regarding C# and Python. Just for the fun of learning. Most of this data is available online and I will put the link to the original sources.
Part 1 of that series is available here.

The following entirely comes from an article by Alexander Yakunin, former Quora engineer and now working at ServiceTitan in San Francisco. I love how Alex goes into the details with his explanations. Kudos!
———————

How does Python compare to C#?

  • The answer is huge, but (hopefully) quite comprehensive. I programmed on C# / .NET for almost 10 years, so I know it really well. And I program on Python at Quora for ~ 7 months now, so I hope I know it pretty well.
  • Python is winner in: ease of learning, cross platform development, availability of open source libraries
  • C# is winner in: standard library, language features, development process and tools, performance, language evolution speed
  • Roughly even: syntax (Python is better in readability, C# has more consistent syntax), adoption.

Syntax

Python pros:

  • Usually shorter notation:
    • indentation, “:” and “\” instead of “;”, “{“ and “}”
    • no need to decorate any identifier with its type (or “var” in C#)
    • no need to specify function argument types and return type
    • no need to nest each function into a class
    • no need to use public / protected / private; you should follow “_name” or “__system_name__” convention, but it’s anyway shorter
    • no need to type “new” to invoke constructor
  • Generator / list comprehension expressions tend to be very short as well.

C# pros:

  • More consistent syntax and formatting rules. I can’t remember any inconsistencies in C#, but instantly wrote this list of inconsistencies in Python:
    • the fact that assignment expression is not an expression at all
    • old and new class syntax (“class X:” vs “class X(object):”)
    • weird base method call syntax: super(ClassName, self).method_name – it violates DRY twice and doesn’t work for old style classes
    • different formatting rules for = in regular assignment expressions and function arg. spec.
    • “except SomeError, error” syntax: show this fragment to someone who never seen Python and ask him what it might mean
    • exception from common rule in tuple syntax: ( (), (1,), (1,2) )
    • class methods: IMO this feature just pollutes standard polymorphism by adding no any significant improvement; worse, people frequently misuse it (e.g. always prefer class methods to static methods)
    • the way Python deals with self / cls (I’ll discuss this in performance section) looks like a leaky abstraction; this adds noticeable performance impact as well
    • threading.local is one of the worst abstractions I’ve ever seen; C# offers few options here, but the most widely used is [ThreadStatic] attribute turning static variable into thread-local static. But in Python you must inherit your class from threading.local to ensure its instance fields are thread-local. So same instance sees different fields in different threads. The abstraction is bad not just because it looks like a huge hack, but also because it requires you to write more code (additional class) in typical scenario (thread-local static variable).
    • system method names with double underscores on both sides make you feel like you write on C 🙂 (4 additional symbols for any of such names)
  • In few cases – shorter or cleaner syntax. E.g.:
    • lambda expressions and LINQ (vs generator expressions in Python) are shorter and more readable in C#
    • lower_case convention for method / function / local names (i.e. most of names) in Python requires more keystrokes than camelCase convention in C#. Basically, you do 1-2 additional keystrokes per each identifier.

Language features

Python features, that don’t map well to C# features:

  • everything is dynamic. Sometimes this is really useful – e.g. you can add your own “tags” (fields or methods) to nearly any object, or use “monkey patching”  (What is monkey-patching?) to change the behavior of third-party code. It’s arguable if it’s good from design point, but there are cases when it’s really nice to have such an opportunity. You can achieve something similar in C#, btw – e.g. by employing dependency injection, extension methods + weak references, but this is more complex. On the other hand, it makes you think more about the architecture that enables you to make such changes without hacky tricks.
  • decorators: there are equivalents in C#, but Python’s decorators are definitely unbeatable in simplicity and flexibility. Closest C# equivalents: nearly any DI container capable of aspect injection (in particular, based on decorator-like attributes); PostSharp aspects.
  • *args, **kwargs – most of static languages don’t provide such a way to enumerate/pass call arguments. On the other hand, there are lots of ways to process all arguments in C#:
  • yield expressions: yield can both accept and return value in Python, but in C# it can only accept a value. On the other hand, it looks like you can implement a very similar logic with async/await (await accepts and returns a value).
  • Class methods: actually it’s good C# doesn’t have this feature 🙂
  • “with” contexts in Python can process exceptions: __exit__ method there gets information about thrown exception. Unfortunately this is impossible in C#: IDisposable.Dispose() has no any arguments there. The feature can be really useful in some cases – e.g. when you need to make a commit/rollback decision inside block exit code. But you can handle this with nearly the same amount of delegate-based code in C#:dbContext.Transaction(() => { /* code */ }

Python features, that can be mapped to C# (i.e. there are close abstractions):

  • list comprehension syntax and generator expressions: LINQ to enumerable (LINQ in general is way more powerful)
  • generator methods: methods returning IEnumerable<T> and IEnumerator<T> in C#. Actually, this feature is implemented in more solid fashion in C#: result of IEnumerable<T> method can be enumerated as many times as you want, and result of IEnumerator<T> method can be enumerated just once. Now look how it works in Python:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
In [1]: def gen():
   ...:     yield 1
   ...:     yield 2
 
In [2]: g = gen()
 
In [3]: [x for x in g]
Out[3]: [1, 2]
 
In [4]: [x for x in g]
Out[4]: []
 
In [8]: g = gen()
 
In [9]: for x in g:
   ...:     print x
   ...:     if x == 1:
   ...:         break
   ...:
Out[9]: 1
 
In [10]: [x for x in g]
Out[10]: [2]
  • lambda expressions: the same in C#; you can get ~ AST of such an expression in C# too. As I’ve mentioned, C# notation for lambda expressions is shorter: “lambda x: x + 1” in Python is “x => x+1” in C#
  • dynamic typing: dynamic keyword in C# + DLR infrastructure. IronPython/IronRuby are built on it.
  • runtime code generation / parsing / evaluation: lots of facilities in C#: lambda expressions, System.CodeDom, System.Reflection.Emit, third-party wrappers helping to implement typical tasks, and finally, compiler-as-a-service in C# 5
  • named parameters and default values: works ~ the same in C# in terms of syntax.
  • tuple expressions (tuple syntax): C# doesn’t provide any syntax sugar for this, but there are tuples (i.e. regular classes).
  • regex syntax: the same.

Now let’s look on C# features, that are missing in Python. I’ll list just the most important ones from my own point of view, but you can get a good imagination of the whole list by looking up this feature-by-feature comparison of C# and Java.

  • Multithreading: Python, in fact, is single threaded: there are “threads”, butGlobal Interpreter Lock (GIL) allows just one Python thread to execute at any given moment of time (except the case when one of threads awaits for IO completion). There are Python implementations w/o GIL – IronPython (.NET-based) and Jython (JVM-based), but since library developers assume there is GIL, you actually can’t rely on GIL absence even on these platforms. Absence of multithreading brings really big performance-related disadvantages for web apps (pretty unexpected, btw, if you didn’t face them) – but I’ll cover this in performance section.
  • Generics and structs: on one hand, Python doesn’t need generics, since it’s dynamic. On the other hand, they frequently provide huge benefits in terms of memory and speed in C# – even in comparison to Java with itstype erasure based generics. E.g. if Vector3D is a struct of 3 doubles in C#, var list = List<Vector3D>(…) will really use ~ list.Length * 3 * sizeof(double) amount of RAM for it, and will be represented by just 2 objects in heap (List<T> + Vector3D[] – the underlying array it uses for storage). But Java should allocate ~ list.Length of objects in heap for this list: each Vector3D is represented by an object in heap there, so totally it uses at least 3 * list.Length * pointer_size more RAM for this. And Python should allocate ~ 4 * list.Length objects in heap (both Vector3D and double are in heap there).
  • LINQ: it isn’t the same as list comprehension in Python:
    • List comprehension is specific syntax sugar available just for iterables in Python
    • LINQ is syntax sugar as well, but it isn’t bound to a particular generic type: it works for any generic type meeting few expectations. In particular:
      • If this type is IEnumerable<T>, it’s “LINQ to enumerable” – a set of extension methods operating as list comprehension in other languages
      • But if this type is IQueryable<T>, it’s what normally referenced as LINQ – a technology allowing to transform a query expression on C# to query on virtually any other language. In particular, it’s LINQ to SQL, LINQ in Entity Framework and so on.
      • Finally, underlying type can be your own. Few examples: LINQ to observables (Reactive Extensions) and parsing monad.
  • Interfaces. They’re really good, if the same contract is implemented by several classes.
  • Extension methods: pretty nice feature, actually, although it’s just a syntax sugar improving code readability. Anyway, I used them in C# (mainly to add helpers to some built-in and third-party types like IEnumerable<T>, IRepository<T> and some enums), and their absence in other languages makes me a bit disappointed. You can use “monkey patching” in Python, that’s something very similar, but not the same: extension methods don’t change the original class and are applied if you import a namespace declaring some of them. So different code can import different extension methods (even if they have identical names).
  • Enumeration types. They’re implemented super-well in C#, and it’s quite good to have this feature integrated.
  • Method overloading: no analogue, but you can handle different argument sets differently in the same method in Python (although usually this is more complex). Method overloading is something natural for static languages, but quite unnatural for dynamic ones.
  • More native code, less interop / plumbing code. C# is fast, so normally you deal just with C# while writing on it. The same isn’t valid for Python: it’s a bad idea to implement nearly any algorithm requiring huge amount of RAM or CPU resources in Python; you’ll quickly learn that nearly any code requiring high performance must be written on C++. All this implies that:
    • If you develop a realtime web app on Python, I’m 99% sure you’ll end up writing few modules on some other (faster) language – e.g. C++. Maybe it’s just 1-2% of the codebase, but it’s +1 language to learn and +1 dedicated subsystem to maintain. Besides C++, you need to know how to interact with C++ code from Python, i.e. how to write some plumbing Python code enabling the interop.
    • Compare this w/interop in C#: I suspect 99% of web apps written on C# don’t  have any C++ code at all, because it’s ~ as fast as C++; there are few pretty rare cases when you need C++, but they’re really rare. And even if you’ll ever need this, C# provides really awesome interop capabilities.
  • async/await, Task Parallel Library and Parallel LINQ: they’re really useful in C#, and honestly, most of other languages don’t offer anything similar in terms of usability. But since Python doesn’t support concurrency, this is even worse there. Probably, parallel processing in Cython is one of the simplest options available, but even it looks more as workaround having really limited application scope rather than a full solution, and AFAIK, we never used it practically.

Standard library

C# definitely wins here: its BCL is:

  • better designed (more classes, less static functions)
  • strictly follows naming guidelines (there are lots of names violating PEP8 in its current class library – they’re there mainly for compatibility)
  • better documented
  • and finally, it offers more abstractions.

I can’t say Python BCL is totally bad, but it really look worse in comparison to .NET.

Development process and tools

I used:

  • C#: Visual Studio .NET (2005, 2008, 2010 and 2012) w/ReSharper
  • Python: latest PyCharm, Vim and some Unix tools like grep

Python pros:

  • Interactive Python. You can test lots of small things instantly.
  • Fast startup / restart. Even such a large application as Quora starts in ~ 5 seconds on my devbox. That’s actually a very good property: it doesn’t add a significant friction if you used to do small iterations.
  • No need to compile the project. More precisely, you _almost never_ need to compile the project: I estimate nearly any complex project has some non-Python code + some parts requiring Cython (Python-to-C compiler).
  • No need to maintain project files. I mean *.csproj and *.sln for C# – usually they’re managed by Visual Studio .NET, but almost any complex project requires some of them to be modified manually.

C# pros: most of C# pros are based on fact that it is statically typed, so development tools can rely on this. I’ll list just the most important

  • Correct intellisense suggestions. I’d estimate that PyCharm provides a correct suggestion only in 30% of cases.
  • Correct refactorings. Nearly any refactoring in PyCharm requires manual fixes. I’d say PyCharm dependency detection gives some false positives in 30% cases, and false negatives in 10% of cases, so you always need to supervise this. And that’s really annoying in case of really large refactorings (100+ usages). Most of Python developers use grep for this, and that’s way more complex. I’d say, that’s probably the biggest disadvantage: you can easily spend 10+ minutes (in some cases – an hour or so) on actions that take 1 minute in ReSharper.
  • Way better “online” error detection / highlighting. The most typical example: if PyCharm is incapable to derive the type of x, don’t expect it highlights x.soemthing as an error. The same never happens in case with C# (of course, if you don’t use dynamics).
  • ReSharper provides way more helpers – mostly, stub generators. E.g. “implement /extract interface” or “implement equality members” is something I used to. PyCharm has nothing similar (probably, because Python doesn’t support interfaces 🙂 ). But anyway, the point is: despite the fact C# requires you to write more bolerplate code, nearly all typical code can be quickly generated by tools like ReSharper. So you actually don’t feel any friction about that.
  • VS.NET provides better support for related tools and technologies. Full support of e.g. Razor ASP.NET MVC templates (intellisense, navigation, errors, etc.), ASP.NET MVC itself and languages like Less/Sass is really useful: you do way less actions to add somethign standard.
  • Compilation detects may be 50% of errors even before launching the code. Yeah, that’s one of benefits of statically typed language: you don’t need to run the code to detect pretty large part of errors.
  • Better debugger, integrated profiler. I can’t say PyCharm is bad here (no profiler, but its debugger is very nice), but I have a feeling that VS.NET offers more useful options.
  • Relatively fast compilation and startup. “Relatively”, because it is way faster than e.g. GCC; startup is fast if you pay attention to this – actually it’s pretty easy to make it slow, especially for a large app.

Overall, I think C# is definitely a winner in scope of this section. Python clearly gives some benefits if your codebase is relatively small, but most of them turn into disadvantages when it becomes large (or huge). E.g. pretty big amount of friction associated with refactoring makes developers to postpone this, that eventually increases technical debt. I can’t say if snowball effect is highly probable in this case, but at least its probability seems higher.

Performance

Disclaimer: I never studied CPython code, so a part of further statements explaining how it works internally might be false: I reason about this mainly by applying my Python debugging experience + general logic. Nevertheless estimated timings provided here are well-aligned with actual measurements, so hopefully, if there are some mistakes, they aren’t vital.

That’s the most painful part of comparison for Python. CPython (most widely used Python interpreter) has a set of issues related to performance, and I’ll try to cover most important of them here; many of such issues are related to other dynamic languages, but definitely not all of them.

It also worth to mention there is PyPy claiming to be almost 6x faster than CPython; on the other hand, so far we couldn’t reach any speedup by running Quora on PyPy. I can’t fully explain why, but I feel this is mostly because:

  • We use Cython to improve performance at all major hotspots; it’s tiny % of codebase, but it looks like we’d have 30-50% worse performance w/o Cython. So CPython our codebase actually isn’t a pure Python code
  • It seems PyPy isn’t quite efficient for applications having huge codebase and large working set
  • We didn’t try to implement any PyPy-specific optimizations in PyPy branch.

Currently performance of PyPy branch is ~ the same or slower as performance of primary CPython branch; I’ll update this section if there are any changes.

So why CPython is slow? Let’s start from Alex Gaynor’s presentation:

It worth to look all the slides, but I’m going to describe what actually happens in CPython on this particular example. Fast-forward to slide 24:https://speakerdeck.com/alex/why…

Alex lists 3 allocations done by this code, but reality seems to be way worse – it’s actually pretty tricky to even list all the stuff involved there:

Python: int(s.split(“-“, 1)[0])

a) s.split(…) involves:

  • a dictionary lookup for “s” name. Few dictionary lookups, if this happens inside a closure (or few nested ones).
  • a dictionary lookup for “split” method of “str” type
  • creation of a bound method object even before call, i.e. a allocation of a new object in heap:
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
In [3]: x = "a".split
In [4]: y = "b".split
In [5]: x is y  # “is” performs reference comparison in Python
Out[5]: False
In [6]: z = str.split
In [7]: z is x
Out[7]: False
In [8]: x1 = "a".split
In [9]: x1 is x
Out[9]: False

This happens because methods are actually descriptor objects in Python. So when is executes some_object.some_method(some_value), actually few dictionary lookups + few allocations might happen:

  • CPython looks for “some_method” in “some_object”’s dictionary (actually, its class object, but let’s assume we still have constant lookup time assuming inheritance hierarchy is tiny)
  • And tries to get __get__ member of this object – to check if it’s a descriptor or not, so probably it’s +1 lookup (actually I hope system methods invoked by CPython itself are invoked ~ like virtual methods in C++, i.e. it’s fast). Since some_method is instance method, there is __get__ member
  • CPython invokes this __get__ member. To invoke it, it must construct and pass a tuple with its arguments. Quite likely, +1 allocation – it’s pretty unlikely they’re passed as structs on stack.
  • __get__ should return a bound method instance, i.e. it’s +1 allocation in the best case as well.
  • Likely, some_method call implies allocation of a new tuple object containing all passed arguments – i.e. (some_value,) in our case
  • Since we invoke a bound method, there can be two tuples: normally bound method adds self to a tuple passed to it, i.e. produces another tuple: (self, some_value)
  • Probably this isn’t intact for methods that don’t accept **kwargs, but the ones that do require a new dictionary to be constructed (pretty large object).

So:

  • instance method access =~ 1 dictionary lookup + 2 allocations/deallocations + 1 VMT-like lookup (__get__ member search)
  • instance method call =~ at least 1 dictionary lookup + 4 allocations/deallocations + 1-2 VMT-like lookups

b) split(…) result is a list, which is usually composed of two objects: list wrapper + array, that’s re-allocated when list reaches its current size limit. So if it returns a list of 2 strings, there must be at least 4 new objects in heap (list, wrapper + 2 strings).

c) …[0]: works nearly any other method call. Fast languages allow to turn off bounds check for such ops, but effect of doing this in Python must be negligible in comparison to other expenses.

d) int(…):

  • a dictionary lookup for “int” name. Few dictionary lookups, if this happens inside a closure (or few nested ones).
  • likely, one tuple allocation (for method call arguments)

So let’s count the minimum number of dictionary lookups and allocations/deallocations for this simple code:

  • Dictionary lookups: 1 + 1 + 1 = 3
  • Allocations/deallocations: 4 + 4 + 4 + 1 = 13

Quick measurement shows that Python needs at least ~50ns for dictionary lookup, and ~50ns for allocation. It’s more difficult to measure deallocation time, but it’s safe to assume it is comparable to allocation time. So expected performance limit of this piece of code is:

  • 50ns * 3 + (50ns + 50ns) * 13 = 1450ns

Checking with %timeit:

1
2
3
4
5
In [1]: s = '1-'
In [2]: int(s.split("-", 1)[0])
Out[14]: 1
In [3]: %timeit int(s.split("-", 1)[0])
1000000 loops, best of 3: 1.80 us per loop

So actual time is 1800ns.

Now let’s calculate what time the same operation could take in C#:

C#: int.Parse(s.Split(‘-‘))[0]:

  • 3 static class, 0 virtual calls. Static call require ~ 1ns in C#
  • 1 string[] allocation, if string.Split is optimized pretty well; small object allocation require ~ 10ns in C#
  • 2 string allocation, if Split produces 2 strings
  • No need to count deallocation time, since it’s nearly zero for any short-living objects in languages with true generational GC w/compactions.

So this code should require at least 33ns in C# (I’ll add actual measurement result later here). Actually it should take a bit more, since Parse and Split do some job, and it should be probably comparable to these 33ns in terms of CPU consumption time. But even this simple calculation shows this code must be ~44x faster on C#; likely the same is valid for other similar languages (Java, Scala, etc.).

So let’s summarize some of CPython issues related to performance, that were exposed here:

1. Lots of allocations / deallocations, that are costly

Thanks to:

  • Bound methods, and likely, a convention of passing arguments as a tuple
  • The fact that nearly any variable (even local) is stored in heap
  • Absence of generics and structs. On contrary, these two things together make C# code almost as efficient as C++ in terms of RAM consumption.
  • Finally, CPython uses ~ regular memory allocator. This mean that it should find a large enough space in heap to allocate the object and track the fact this space is used, and do the opposite on deallocation. This happens in any case, i.e. for small, short-living objects as well.

Now imagine, .NET needs just to move the pointer + clean up allocated block for any typical allocation, and does virtually nothing for typical deallocation (really: Garbage collection (computer science)). So .NET heap acts much more like a stack: there is no need to look up for free space to allocate some RAM, and no need to worry about deallocation, since there is GC, that touches just live objects, and knows absolutely nothing about dead ones. And since most of objects are short-living, .NET spends zero resources on deallocation of most of objects. Certainly there is amortized deallocation cost, but it’s way smaller than in Python (may be it’s smaller even than in C++, btw) due to this fact.

This explains, btw, why Java is nearly as fast as C#, although there are no structs and true generics: this stack-like heap behavior makes allocations/deallocations of small objects really cheap – almost as cheap as placing them on regular stack (structs and other value types in .NET live on stack). So basically, Java may be slower due to this only when lots of such objects are long-living, so they’re promoted to higher GC generations, and this adds some pressure on full GC cycles. This affects on CPU cache hit ratio as well, since any object in heap has additional header (16 bytes in 64-bit processes on .NET; not sure about Java), so if objects are small, % of excessive data in continuous block of RAM can be pretty high – e.g. at least 80% for such blocks of integers. But anyway, that’s Java; Python has way more serious issues here.

2. Relatively large objects

Most of objects are dictionaries, so they tend to consume several times more RAM than e.g. in objects C#. Efficiently this reduces the size of CPU caches by the same factor. But L1/L2 misses are super-expensive – they can easily slow down your code by an order of magnitude, and Python programs tend to get these 10x way easier.

3. Bad garbage collection (GC) implementation

CPython doesn’t have true generational GC. That’s mentioned in 1), but here I’d like to show few more side effects of that:

  • It never moves objects in memory (Do python objects move in memory during execution?), thus there are no “true” generations defined by border addresses in RAM (I suspect generation is just some mark in object’s header there)
  • It uses reference counting + likely, mark and sweep-like collector to find and remove garbage
  • There are generations, but it looks like they aren’t quite helpful: GC pause can easily take few seconds, if you have a large heap

Why it’s bad?

  • No compactions = RAM fragmentation + bad CPU cache utilization. For comparison, .NET tunes up GC in such a way that Gen0 is always inside L1 CPU cache, and Gen1 is inside L2 CPU cache – i.e. most frequently accessed objects are cached quite efficiently in C#, and almost always aren’t in cache in Python – due to RAM fragmentation and significantly larger average object size. Details: Garbage collection (computer science)
  • Reference counting = global interpreter lock = no true multithreading + no benefits of fork()-ing the process to share a single copy of code and initial data set across several Python processes
  • GC pauses are proportional to the amount of RAM used by your application. Thus CPython app using multiple GB of RAM and running without pauses of several seconds (if not minutes!) is something impossible. Actually even an app using about hundred of megabytes requires special GC tweaks  and manual GC cycles to avoid pauses at random moments.

Finally, I found it quite misleading that “gc” module documentation doesn’t reflect this: 27.12. gc – Garbage Collector interface – Python v2.7.5 documentation. Worse, if you look on descriptions there, I bet the initial impression you’ll get is: “cool, it has generational GC!”. There are parts that might make an experienced developer to suspect the opposite (can you find them, btw?), but I feel like the only conclusion most of people can make is that Python has nearly the same implementation of GC as other modern languages. And that’s misleading.

4. Lots of dictionary lookups

  • Python uses dictionary lookup to resolve address of nearly any symbol (even local variable). Dictionary lookup time is ~ 50ns there (btw, it’s nearly the same in C# for Dictionary<K,V>).
  • Static languages (including C#) resolve virtual method addresses usingVirtual Method Table lookup, and such lookup usually takes 2-3ns to complete.

.NET code uses dictionary lookups just in one case: when it invokes virtual generic method parameterized by generic type(s) (i.e. the call looks like someObject.SomeMethod<T1,T2,…>(…)) – dictionary lookup is the only option it has in this case. But this is relatively rare thing, actually – especially for hotspot code. And even this dictionary lookup usually takes ~ 20ms there.

5. Multithreading and related issues

As I wrote before, Python, in fact, is single threaded. And just one this things makes it impossible to implement e.g. such optimizations available for web applications on .NET:

  • Any in-memory caches: you can share them across all the threads serving web requests on .NET, but you can’t do the same in Python, since there is no point to handle more than one web request in a single Python process (if it doesn’t wait a lot of time in IO, of course). So you can assume each web request handler written in Python is actually a process with its own address space, and thus you can’t share caches across such processes. So if you run e.g. 16 Python processes on web tier machine, you can assume that effective cache size is ~ 1/16 of cache size for the same .NET app. You can try to tackle this issue by running a process maintaining shared cache for multiple Python processes, but you’ll get additional interop expenses (serialization and deserialization), that add a huge overhead in comparison to direct memory access.
  • The same is actually related to Python code itself. Imagine you run a large web application, that allocates 100MB right on start for code and data. So if it’s .NET application, it can utilize threads to service concurrent requests, there is just one instance of these 100MB of shared stuff. If 5-10MB of it is the part that’s used most frequently (working set), you have a good case, since it’s close to L2 CPU cache size. Now imagine the same in Python: 16 similar processes allocate 1.6GB of right on start; working set size is 80-160MB, which is far beyond L2 cache size, so just one this thing slows down everything by may be a factor of 10. Probably you think you can fork a single Python process to ensure there is a single copy of these 100MB, but this won’t help much: CPython mixed garbage collection utilizing reference counting, so basically, if you reference / dereference something (e.g. copy a reference to some object into local variable), this counter changes its value. So any memory page containing any object from this hot set has nearly zero chance of staying unmodified – i.e. nearly all pages after forking will be copied pretty fast by each of such processes.
  • Above issues are way more painful for Python web apps than e.g. absence of TPL, PLINQ and async/await analogues: you get parallelism almost for free here, so additional parallelism is actually rarely necessary. But this “free parallelism” in .NET is actually way better then “free parallelism” in Python.

Cross-platform development support

Python clearly wins here: C# works very well on Unix under Mono, but it’s mostly about its base class library. Nearly anything tightly bound with Windows isn’t available there. Incompatibility map: Compatibility – Mono

Adoption

Roughly even: PYPL PopularitY of Programming Language index – pyDatalog

Availability of open source libraries

Based on my experience, Python and C# roughly even in terms of availability of free / open source third-party libraries. Nearly all you need to develop a web application is free on .NET.

But Python has definitely more open source projects on GitHub: Top Languages · GitHub (many C# projects are hosted onhttp://www.codeplex.com/ – originally it was the best place to host your own C# project, but right now it’s GitHub, so I think it’s fair to ignore this).

So Python is winner here.

Ease of learning

Python seems way easier to learn:

  • Basic syntax requires you to know less language constructions – e.g. no program in C# can be written w/o declaring a class; you need to know what’s compilation, assemblies, namespaces, classes, methods, public/private/static keywords, etc. On the other hand, you can write a program even without declaring a function in Python. So it’s easier to learn Python iteratively: you need to know almost nothing at start, and use more and more features while studying it deeper. In contrast, C# requires you to learn way more before you even start to write your first program on it.
  • Interactive Python provides really nice way to learn the language and run quick tests.
  • Python standard library is mostly built over functions – there are just few classes, no any complex inheritance, etc.; in contrast, C# base class library is fully object-oriented: lots of classes, sometimes – deep inheritance. Moreover, some parts of it require you to understand functional concepts very well – e.g. as I wrote, LINQ in C# is way more powerful than list comprehension syntax in Python, but that comes with associated learning expenses: list comprehension syntax in Python is, in fact, just a nicer way to write loops, and thus it’s super-easy to explain this. And LINQ is, in fact, a syntax sugar for defining Monads in C# + implementations of some of them (IEnumerable<T>, IQueryable<T>). This description highlights the difference very well 🙂 And LINQ isn’t the only example: there are few other parts of C# BCL that require you to deeply understand all the concepts; try learning WPF, for example.
  • You need to know a set of specialized tools to write on C#. E.g. I used Visual Studio .NET with Resharper and a set of other plugins, Far (it’s like Midnight Commander, but for Windows), Redgate .NET Reflector and IIS on daily basis. And most of people writing on Python use just Vim/Emacs + a set of standard Unix tools. Not sure what’s better here in terms of learning curve – basics of VS.NET are pretty easy to learn; on the other hand, a typical Unix developer doesn’t need to learn any new tools at all to develop on Python, assuming he already knows Vim and unix tools like grep. But… Typical Windows developer knows VS.NET as well 🙂 Anyway, the point is: there are simpler (i.e. less advanced than for C#), but more generic development tools for Python.

Language and runtime evolution speed

I feel like C# evolves way faster:

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: